Saturday, March 20, 2010

How Big is Your Code Base?

This morning someone asked me how big Scrivener’s code base is, and I didn’t know how to answer - yet it’s something I’ve been wondering myself. It comes up occasionally, usually in two situations: 1) Someone writes to me wanting to know how much code it took to create Scrivener, because they have an idea for a program and wonder what’s involved; 2) Someone is insisting that I could port Scrivener to platform X or “just add” Y “easily enough” and is getting uppity that I’ve said no, so I want to puff out my chest and huff, “Look, I have written six billion lines of code mostly unique to the Mac platform and I’ve already got grey in my hair - coding Scrivener ain’t like dusting crops, kid!” (These exchanges usually end with me apologising for something.)

Of course, how much code is behind any given program is misleading, and to a large extent rather meaningless, too. For a start, any Cocoa program stands on top of the vast amount of code written by Apple, so counting how much code you have written for a Cocoa application with a view to calculating how much work it would take to port it to another platform (even the iPhone or iPad) would be pointless, as it would not include all the extra code you might need to write to replicate features provided in Cocoa by Apple. Also, small programs with less code are often better for particular jobs than large programs with reams of code. For instance, Bean contains much less code than Microsoft Word, but it’s much nicer to user for basic documents. On the other hand, a good program that appears very simple to the user may contain a great deal of code that checks for potential errors. Besides, from a programming point of view, there’s nothing better than cutting code. It’s always a good feeling when, upon reviewing a section of code you wrote ages ago, you slap your forehead and realise that you have written 50 lines that could have all been done in one. (There’s a great story about an Apple Lisa programmer who, when forced to fill in a form declaring how many lines of code he had written that week for the sake of proving progress to the management, wrote “-2000” - they didn’t use the forms much longer.) Less code is usually better, but sometimes more lines of code is necessary for the sake of readability. You might be able to cram ten lines of code into one, but if you come back to it two years later and have no idea what that line is for, then what was the point? It would have been better to have written ten lines of code with well-named variables, or at least to have made more space for meaningful comments.

Still, caveats aside, after having worked on a project for more than five years, it would be interesting to know how much code you have ended up with. Yes, it’s meaningless - the results of your labour are measured by the quality of the application and not by the quantity of code - but the obsessive compulsive, anal retentive in me wants hard figures (whatever a “hard figure” is - a die-cast Arnie?).

How do you measure the size of your code base, though? It’s not as simple as it at first sounds. Do you count the number of files you have generated? But then, in object-oriented programming, each class has at least two files, and sometimes more if you have created categories. So do you just count the number of different class files you have created? Each of those could contain wildly different amounts of code, though, so maybe you should could the total lines of code in all? But then, even counting lines of code is far from straightforward. For a start, there will be lots of blank lines - so what about if you count only the lines of code that aren’t blank? But what about lines that don’t contain any statements, just an “if (something)” or an end bracket? Okay, so how about we just count lines that have a semi-colon in them, as that indicates the line has a statement in it. On top of all of this, though, when thinking about text, I like to think in word and character counts - if I’d been writing instead of coding, how many books could I have written (or, more accurately, agonised about not writing)?

Well, in the name of procrastination, I decided to spend this morning writing an application that would count all of these things, as it seems to me that they all have something to say about how much code you have written. There are a few other apps out there that will count lines of code and so on, but after spending twenty minutes searching, I figured I would just write my own. I called it “Xcode Statistician”, and if you are a programmer you can run your own Xcode project through it by downloading it from here:

Here are the results of running the Scrivener 1.5 code base through it:

Blimey. That’s quite a lot of code. But before I pick that apart a little, here are the results of running the Scrivener 2.0 code base through it:

Blimey O’Riley - now that is a lot more code. In fact, the code base has doubled - Scrivener 2.0 has twice as much code as Scrivener 1.5. Perhaps this shouldn’t come as a great surprise to me, as I’ve been working on Scrivener 2.0 for over two years now (and it will be out later this year, folks, I promise). And yet it was a surprise, because Scrivener 2.0 is designed to be simpler to use than Scrivener 1.x - I’ve taken on board a lot of user feedback and had time to redesign elements with which I wasn’t happy, and I really feel it is more accommodating to new users. But then, I suppose making things look simple can sometimes take a lot of code. (Not all of that code is mine, of course - a small fraction of it will be source code I’ve used from elsewhere.) And then, I have written a plethora of new classes, from a text comments view to a collections view, I’ve separated code out into different files to make it more manageable, I’ve significantly enhanced the functionality of the corkboard, and so on - so I suppose it makes sense that I have written so much code in the past couple of years.

Still. It should give some indication of just what I mean when I say that porting Scrivener - even a putative “simplified” version - to the iPad or to any other platform would be a big job. Sure, in comparison to something like PhotoShop or Word, Scrivener’s code base is puny, but I think it’s a fair whack of code for a single developer. (And I don’t say this as a boast - if anything, the figures depress me, because that now means twice the amount of code to maintain and double the fun when it comes to tracking bugs.)

So, to answer the question I was asked this morning: How big is Scrivener’s code base?

Approximately 200,000 lines of code comprising at least 70,000 statements.

600 class files comprising around 230 custom-built classes and 40 classes that extend the existing Cocoa ones.

Nearly 600 image files. (Even accounting for those sourced from elsewhere, how on earth did I manage that? I’m rubbish with Photoshop - no comments about icons you don’t like, please.)

50 interface files.

Over six million characters.

But here’s the real kicker: 600,000 words - of course that’s only rough given that many of them will be numbers and variables and single characters, but all the same. Had I the brains or creativity or drive to do the thing that drove me to create Scrivener in the first place (still with me?), I could have written about six 100,000-word novels.

In fact, I think that is how I’m going to quantify it, as it seems wholly appropriate:

How large is Scrivener’s code base?

Answer: About six novels.

19 Comments:

Blogger Unknown said...

Entertaining and instructive as always, Kevin. As you discovered, you're actually a prolific writer - your "novels" are just written in Cocoa. Cheers.

5:52 pm  
Blogger Marcia said...

I wish you'd put all this blog-writing energy into a iPad version of Scrivener. I think it would be a huge seller.

12:47 am  
Blogger kayembi said...

Hi Marcia... Er, thanks. I'd rather put my energy into a novel, though, sorry. Please see the forums and my previous posts on the iPad, as I've sort of discussed it to death, but basically: 1) The iPad doesn't have a rich text system (Apple wrote their own for Pages and it's not available to devs; I'd have to write my own - have you any idea how many years that would take?); 2) The iPad is a completely different platform and I have no intention of starting over on another platform just as the Mac version is getting where I want it; 3) Is the iPad *really* a serious writing environment? I have my doubts. Certainly I don't see myself trying to hammer out 3,000 words a day on one, and if I'm not going to do any serious writing on it myself, I'm not interested in developing for it - I'm only interested in developing software I'm passionate about and that I'll use myself (life being too short); 4) I just can't get that excited about an oversized iPod Touch - I know that makes me a freak among Apple-users, but that's just how it is; 5) As a single coder with a family and literary pretensions, I have no intention of spending the few spare hours I get outside of developing the Mac version on developing Scrivener for another platform; 6) Did you even read my comments above about just *HOW BIG* Scrivener is and how much it would take to translate it to a different platform?; 7) Do you realise how few of the controls and interfaces that Scrivener is built around are available in the iPhone OS? 8) I'm not a code monkey; 9) Hippy that I am, appeals to my capitalist side do not work (though promises of alcohol do); 10) This is the last iPad heckle I'm responding to, to prevent spontaneous combustion :) 10) It's now by birthday. 11) I'm going to bed.

1:09 am  
Blogger kayembi said...

P.S. I could have written ten lines of iPad code in the time it took to write that.

1:10 am  
Blogger kayembi said...

12) I can't count.

9:55 am  
Blogger kayembi said...

Bryan - thanks! But why does everyone insist on calling me Kevin? :)
All the best,
Keith

9:56 am  
Blogger Unknown said...

Hi Keith. I have no idea where Kevin came from when I typed that. I offer in trade that you may call me Mauricio, or something equally not-Bryan. Seriously, though, great posts about the inner workings of a fantastic app.

7:36 pm  
Blogger Emilio A. said...

Great. Post. Keith.!!!
(and yes, you are a great Cocoa "novelist". Henry Miller comes to mind. Tropic of Scrivener haha!!)
Cheers.
Emilio

4:42 am  
Blogger Rachel Blackman said...

I feel your pain here. For my day job, I've had people go 'Okay, you got the iPhone version out... so you'll do Android next, right? That can't take too long.' (Sure, porting my Objective-C codebase to Java and a completely, vastly different UI library and usage model will be a snap, folks. I mean, I don't need to sleep, right?)

So rather than asking for an iPad version, I've got a different question: have you considered just documenting the Scrivener file format? (Yes, it's fairly easily reverse-engineered, but still.)

As there are other Cocoa developers out there who also use Scrivener for their non-code writing (me, for one, and I'm hardly the only one), and that would probably encourage someone else to make a 'Scrivener Companion' type program for iPad. Just something you can drop a Scrivener project onto (via the iTunes import/export files interface in iPhone OS 3.2 and later) and do some basic editing of docs and referring to notes while out and about. Then transfer back to the big, full Scrivener once home.

(Or possibly it'd just encourage someone to add support for Scrivener bundles to an existing portable text editor for iPhone/iPad. Either way, same basic end result.)

6:47 am  
Blogger kayembi said...

Hi Rachel,

Thanks for your thoughtful comment - it's good to know I'm not alone! Funnily enough, I am right now engaged in documenting Scrivener 2.0's file format. Scrivener 1.x's file format was born of a "whatever works" philosophy, whereas I've spent some time designing a decent format for 2.0. It will still be a file package, but many of the main files now use my own custom XML rather than Apple .plists, and the RTFD files have become RTF files to make them less platform-dependent. The specs are somewhat mammoth (over 100 pages), but after 2.0 is released my intention is indeed to make them available on the site so that other developers can read and write Scrivener files if they need to (there's support for maintaining custom data added by third-party apps, too).

I have been speaking to a couple of iPhone developers about possible back-and-forth with existing programs, too, but there have been a number of technical difficulties on both sides so far. Ultimately it would be great if an existing app can sync Scrivener documents or packages, until such time as we might be able to do something ourselves.

9:13 am  
Blogger Terry said...

I love Google! I was just looking for a tool that does what "Code Statistician" does and found your handy dandy little tool in a few moments.

Thank you so much for building this. Have you considered publishing the source? I would love to extend it to differentiate between comments and non-comment lines. (Yes, I am fully aware of the "are comments SLOC perpetual debate")

Thanks again

T.

1:46 pm  
Blogger kayembi said...

Glad it's coming in handy for another programmer! You're right, I should put the code up on our "Free Stuff" page (http://www.literatureandlatte.com/freestuff.html), and I will do when I get chance. As that will probably be a little while, though, feel free to drop me a line on contact AT literatureandlatte DOT com and I can send you the Xcode project if you like.

10:05 pm  
Blogger Rachel Blackman said...

Revisiting this post, I thought I would actually share how my iPad has become part of my Scrivener workflow. My solution is a little odd. :)

I actually wrote something to sync a Notes folder in Scrivener to an Evernote notebook named for the Scrivener project. Then if I am at the coffeshop writing, I can be in full-screen mode on the MacBook Pro, but still have all my notes accessible in Evernote on the iPad. Conversely, I can edit notes on the iPhone or iPad in Evernote, sync the desktop and have them slurped back into Scriv as the Notes folder of the appropriate project.

I have come to realize that even with an external keyboard for the iPad, I am not likely to use it as my primary writing device. But I do find the ability to tweak the notes/reference/brainstorming portion of a doc on-the-go increasingly invaluable, when an idea strikes. Linking my .scriv file Notes folder to Evernote means I can tweak 'em from iPad, iPhone, Android, Palm Pre, whatever. Problem solved!

I mention this in the vein of our earlier discussion about documenting the file format because if you haven't taken something like this into account, I'd like to ask that you do. I would love to update Scrivernote to handle 2.0 docs in a less horrible way than I do 1.x docs. (And preferably handle Evernote stuff via their sync process rather than writing into the local cache, which is a whole other story. Scrivernote is pretty much a horrible 45 minute hack I am both ashamed of the methodology behind and still find invaluable.)

9:24 am  
Blogger Joshua Mostafa said...

As I'm always saying to clients, simple for the user means complex for the developer. No matter how many times I say it, they seem to conflate the two opposing kinds of simplicity. I end up feeling like a stuck record :/

On the comparison of writing fiction with writing code, in my experience they are diametrically opposed in where the time goes. With fiction, I find I begin very slowly and speed up as I warm up to it. With code, at first it flows very eaily and then slows down as I discover problems I hadn't considered ahead of time. Cutting is very important to both though :)

11:51 pm  
Blogger kayembi said...

That's a very good point on the differences between writing fiction and code - I find exactly the same thing. I can dive into writing code, knowing what needs to be done, very quickly, but then there's often that horrible moment when you realised you forgot something and spend an hour trying to think of a way of not starting over... With prose I end up staring at the page for a long time before starting (often months :) ).

As for the illusion of simplicity, I think it's sometimes easy for users to assume that because, say, Apple have done something in one of their apps, then it must be easy for everyone else too, when the opposite is often true. The most technically difficult user suggestions often start or end with the phrase, "This would be really easy to add..."

5:24 pm  
Anonymous Anonymous said...

@Rachel
Care to share your scrivenernote hack? it is exactly what i've been trying to figure out how to do and for uses much like what you describe, but lack the programming knowledge..

3:41 pm  
Blogger Rachel Blackman said...

@NES - The Scrivernote code is pretty nasty; it's a 45-minute hack job run mostly via hand-tweaked config files; no actual UI or usability.

The concept is perfectly sound, the code just needs to be something less... fragile than my little personal-use hacky tool. For instance, a Scrivener -> Evernote sync is touched off by a Dropbox sync notification, while Evernote -> Scrivener updates have to be done manually, and the process can break as a result. Etc.

I will, however, see about trying to clean it up in the next couple of weeks to be usable by people-other-than-me and toss it up somewhere, if that'd be useful?

5:37 pm  
Blogger Gregor said...

Thanks for the post, as well as for the useful little program. I can safely say that my code base is nowhere near yours in size..

As a dabbler in both programming and writing, I must say that I am impressed with what you've accomplished with Scrivener. I use it almost daily, and recommend it to everyone I know. I have managed to get my mother, who actually is a published novelist (http://www.albertbonniersforlag.se/Bocker-auto/Bokpresentationssida/?isbn=9789100123161), to show interest in Scrivener. Hopefully, she'll get to love it and use it for her next novel.

Sincerely,

Gregor Tomasevic,
Sweden

8:07 pm  
Blogger kayembi said...

Hi Gregor,
Many thanks for the kind words - much appreciated!
All the best,
Keith

7:11 pm  

Post a Comment

<< Home