Log in

No account? Create an account
Previous Entry Share Next Entry
Coding practices
A long time ago in a galaxy far far away, I used to play on a MUD called Lunar Eclipse. (telnet://lunar.betterbox.net:2500 , if it's still around.) Right around the time I started playing, a new coder was brought into the immortal team, Antilochos.

Anti was a cool guy. I was starting out as a Paladin - Anti was also the "Lord of Paladins" or something like that - I don't remember the exact phrase. Anyway, his primary goal in coding seemed to be to bring the Paladins (a notably under-skilled class to start with) into line with the rest of the fairly powerful classes on the MUD.

He did well at this. He coded a lot of nifty features and so on. But in the end, he did that job a little too well. He ended up creating paladins as an over powerful class, and leaving a lot of other things unfixed. One of his tasks was supposed to be fixing the math behind attacks and so on in the MUD - something he didn't end up doing at all, which caused a lot of trouble in the end.

The entire situation reminds me very much of LiveJournal development, right down to the other staff. Elentari was the overbearing head immortal - she paid the bills, and because of it, it seemed like she sometimes acted like she could do whatever she wanted. Domino was the quiet second in command. He always had something up his sleeve - new areas and so on, but it was never big and flashy, and he was oftentimes a peace keeper. Krystnaalt was the justice, bearing trial over crimes committed in game. Antilochos was the new guy who came in, didn't really realize what he was doing, but went about making a lot of big changes - ignoring some of the social mores preventing such big changes.

I can see a mapping here to the LiveJournal staff. Brad is the leader of the pack. In the end, all decisions and so on go through him. Domino reminds me a ton of Jesse - been around for ever, but not flashy, just kind of working in the background. Krystnaalt reminds me of Ferrell - a cool guy who just tries to do the best he's got with what he has. And Anti reminds me a ton of Mark.

Now, here's where the problem comes in. Anti eventually got fired from immortal staff. The reason was simple - he ignored the little stuff in favor of the big stuff. He made huge changes to the system of classes that put the Paladins in place, while leaving the little bugs because they weren't "important enough" in his mind. I see a striking similarity here between the new guy mentality.

Mark has put in a ton of time developing on LiveJournal. There's a bunch of new features because of him - features that a lot of people like and enjoy, and think are really worthwhile. However, at the same time, the bug tracking database fills with bugs. Currently, there are 472 bugs open in the LiveJournal bug tracking database. In one week, I reviewed something in the neighborhood of 21 bugs - on my own, in free time, in addition to having an academic career. Yet it takes weeks or months for employes to catch up to patches. 8 months for a patch to sit - 3 lines of a patch. And it sat for 8 months, until an employee came along and said "That's not good enough, sorry." 5 minutes of review, a minute writing a comment - 10 minutes max of looking, and it took 8 months. Why? Because nobody pays attention to the "shit work" on LiveJournal. The small, idiotic things get left for "volunteers", where they don't get done because, amazingly enough, not a lot of volunteers enjoy doing the niggling shitwork.

In the end, Anti got fired because he didn't take care of the shitwork. I don't think that will happen at LJ, simply because here, the shitwork is passed off onto volunteers without a second thought. It's not a spoken rule, or anything resembling one - it's simply an unspoken rule that seems to be commonly followed.

I see a lot of work being done on new features, new improvements - and not many bug fixes at all. I hope at some point in the future that changes without a volunteer force behind it. I try and be that force, but I'm tired of it. I'm tired of being told I shouldn't consider myself a LiveJournal developer, simply because I don't write the huge patches. A lot of little stuff has gone in because of me - a lot of little changes that people see everyday, and no one realizes. There's a lot of other people out there like me - Zach, for one, and a bunch of others that I can't even begin to name because there are so many.

Sadly, no one seems to care at LiveJournal. 472 open bugs, 300+ of which are non-enhancement, 200+ of which haven't been touched, even with so much as a CC list change, for more than a month.

Hopefully it can change.

Sadly, I doubt it will.

  • 1
I see your point. And perhaps in a world where the situations were the same, you're right, my job might be in danger.

You're forgetting one part of the LJ situation, though. I get paid to do what Brad tells me to do. This morning I got about 3 emails from Brad:

- write a patch for something (bug 1810), and the journaltype thing
- write a new console command for Ryan (needed soon, because without this console command, Brad has to do the stuff manually)
- commit patch to 1809 (Brad reviewed and said to commit)

Then, I had promised the Abuse team to take a look at one of their requests over the weekend, since one of my primary job functions is to do things for Support/Abuse, so I made get_maintainer work with users.

And then, LJBook is going to start supporting comment download soon, so Brad said that he wants a comment exporting interface done "before LJBook is ready for it" -- or, in other words, Soon.

Also, the bugs that this Anti guy didn't fix were bugs he made. Bugs in his code. I don't recall any instances where I introduced a bug and didn't fix it when it was brought to my attention. (If I did, mybad, show me and I'll fix it.)

So, the way I see, as long as:

- I do what Brad says.
- I fix bugs that I introduce.

My job isn't to go through Zilla and fix everything that everybody else thinks should be fixed. My job is to fix what Brad says needs to be fixed. Oh, and if I have any time after that, I can go work on Zilla.

But... in the essence of being the more public dev, I will try to start devoting time to reviewing one or two patches a day that are marked as "patch, reviewed" and submitting feedback and getting things committed if I can.

But, I'll want your help for that. If you care enough about this, each day, find me a bug that has a patch that should fix it (you don't have to review it yourself, mind), and I will go over it when I have some time in the day. Sound good? I'll run this by Brad and make sure it's okay, but I don't foresee any problems with doing this.

Would it be totally out of the question for LJ to have one paid staff member whose job is soley to deal with bugs and patches?

Forgive me if that's already been considered and decided against, but it seems to me like it could be neccesary and helpful.

Granted, I also know approximately dick about how much goes into dealing with code-y stuffs.

I have to admit I don't know much about it either, but as much as that sounds like a good idea, I don't think it's justifiable from a business standpoint.

A full-time employee whose job is simply to go through zilla and handle stuff would run out of things to do rapidly. Maybe it would make more sense to have a volunteer elevated slightly who sort of acts as a gatekeeper to zilla, sorting through the mess and coordinating the efforts of the volunteer devs.

Or that. From your comments around the site, you seem to know more about it than I do.

Hiring someone just for that might not be justified, but sending someone to actually look at zilla as their main task for a while would definitely not be a bad idea. With 10 employees, including several people who just do minor hacking most of the time, it wouldn't be difficult to have those people work on Zilla, rather than new things. David already does this, to a certain extent, and Whitaker did a lot of it this summer. Even with just those two constantly working out of Zilla, things would happen much faster than they do now.

Personally, based on what I've seen, someone could be assigned to Zilla alone and have a job for several months. Getting a decent QA process going could be a full time job - generating a testing harness, running the code in more environments, etc. etc. If nothing else, they could also work on keeping documentation up to date - Jesse's position in that role has failed miserably.

Volunteer effort isn't the way to get things resolved. Developers have a responsibility to ensure that their site works well: when it doesn't, people get upset. That's a fact of life. The fact that no one in LiveJournal development has taken the responsibility to do so is frustrating to me, and a lot of other volunteers in areas like support as well.

I realized as I was writing this that the fact that Anti got fired was a completely different situation. In that situation, the administrators of the situation gave a shit about the bugs that were in the system - on LiveJournal, they don't care until the public notices. Example is clear in lj_biz, bug 172, bug 1718 or whatever that one was: for months (more than a year, in the case of bug 172) the bugs had been sitting around, with no developers devoting any attention to them. Now, they're fixed - because the public cares about them and brings them up. Is that what has to be done to resolve a bug? Bring it up in a public forum with several dozen comments attached?

The bugs that Anti didn't fix weren't his bugs - they were existing bugs in the system when he was brought onto the team. (Hired isn't the right word, nor is fired - because he wasn't paid.) They're bugs that still exist in the ROM 2.4 MUD system. They exist in the base code, and always have - but he was brought on as a developer in part to fix them. This isn't the case here, simply because nobody cares if old bugs are fixed except people who have to deal with them not being fixed.

The problem in the situation is that it's not anyone's job to fix the bugs in Zilla. The problem is that there is no dedicated effort to making LiveJournal work well, only in making LiveJournal work.

I'm not expecting you to change things. Your job isn't to do that. However, I do think it would be nice if the development process at LiveJournal included some kind of standard under which bugs could actually be taken care of. 300+ non enhancement bugs is simply ridiculous, when you consider the number of people working on new features.

I gave up on LiveJournal development on February 25th, 2004. I may return, if I see improvement in the situation, after Easter. I'm a horrible Catholic, but part of my Lenten sacrifice includes me avoiding situations which lead to me fighting with people. Development spurs that - all I end up doing is fighting because I think that the things that don't seem to be important to LiveJournal staff really are important to the LiveJournal code.

You're ignoring most of the point here, which is that volunteer developers are not supposed to be the ones taking care of the bugs. That would be a volunteer effort that they could participate in if they were interested. However, myself (as well as many other developers) aren't interested anymore. The reason I'm not interested is because of exactly what you just pointed out - the expectation that volunteers are supposed to do something.

Now, I don't think it would be a bad idea to bring this up in lj_dev: there are a lot of people actually interested in development still, and that's where to find them. However, I'm not one of those people anymore. I may return, but I need a break. We all do every now and then, and I think after the work I've put in here with absolutely no recognition, it's well deserved. Disagree if you must, but it is my decision in the end.

I wish you good luck. You've got a huge precedent set up against you, and I hope you do well in changing the way it works. I hope it can change - if the attitude of developers actually changed, I might be tempted to work myself back into the development world. However, right now I have no motivation to participate in any way. Good luck in recruiting those who still do.

Now, I don't think it would be a bad idea to bring this up in lj_dev

I certainly do. While there are a lot of people who are still genuinely interested in development, that comm is watched by tons of people who aren't... or who don't have the first clue. And as soon as you open that particular floodgate, there ain't no going back.

I've tried to open a gate to more people before. I posted about #lj_Dev there at one point - you can see the gigantic floodgate that opened. (Yes, I'm slightly sarcastic ;))

The real point is that the people who don't do well at development would quickly be filtered out by those that do, I think. That is, of course, if there was even any response at all - something I'm rather doubtful of, but I'm cynical and jaded to the core ;)

Well, I disagree with you here, really. Posting and saying "hey look, here's an IRC channel where a bunch of people hang out" is completely different from posting saying "hey, we're looking for people to guide the paid devs to worthwhile patches and therefore influence the development process". Every jackass with a keyboard (and you and I both know who I'm talking about) will be ALL OVER THAT.

I think after the work I've put in here with absolutely no recognition

I simply don't understand you. I know that I recognize everybody who contributes to something I commit, and I've posted your name in there a bunch of times, and you've commented to roughly half of them saying "it's not me, don't recognize me!"

Either you want recognition, or you don't. You can't have your cake and throw it away, too. (?!?!) Make up your mind, please. Either you are involved in development (hanging out in #lj_dev, talking about dev stuff, commenting to dev posts), or you're not. But you need to make up your mind.

If you made this original post with the point of motivating change, then you went about it in entirely the wrong manner. But we're never going to see eye to eye on that, unfortunately, so I think that we're not going to have any fruitful discussion on this subject.

I wish you well, but I wish you to grow up, too. I echo sherm's sentiments--you're a fucking smart guy. You need to find something to do with your smarts that will make you happy. It doesn't seem like LiveJournal is the way to go for you.

Not once have I ever heard "Hey, nice job." from someone on the LiveJournal development staff. (Support, of course, is a completely different matter. Positive feedback there is constant, which is one of the reasons that I have absolutely no problem with continuing to work there like I tend to with development.) Not once have I ever gotten a "Hey, could you check this out?" Changelog is a place to put "This is who did what" to keep track of it. Now, I could be wrong in that completely, or in disagreement at the very least, but it's my opinion on the matter. Recognition isn't about a post in a journal - it's about a thank you from someone who matters.

I have gotten thanked by people for work that I've done on LiveJournal - by people outside of development. By danbri, for example, or pne. In any case, the recognition is not my main complaint. I didn't care if I got recognized so long as I got feedback. My comments to changelog were exactly accurate in the cases I made them - coming up with the idea is not being the implemnter for the idea, in my opinion, and the person who actually gets off their ass and codes it is the person that deserves credit in those kinds of posts, because those are the developers, and identifying the developers is exactly what changelog is for.

I was unaware that hanging out in #lj_dev required development work in your mind, or even to be a part of development. If you feel that way, I think that it's a rather silly opinion. There's not a black and white with regards to development - some people have more or less free time than others, that's just the way that life works. If someone's busy or otherwise involved, asking them to participate in development isn't the way to get things done. I've mentioned several times in your view that I don't plan on doing development for a month or so -- I'm sorry you don't seem to have noticed, but I don't think the fact that I don't actively code should mean that I'm not part of the entire development process. I think that's just a silly idea.

I decided a couple weeks ago that LiveJournal development wasn't the way to earn respect. I decided that when I stepped out of the LiveJournal world - and started doing development on projects that actually like developers to help out. I offered a patch to a mailing list - and within 12 hours, had had two people praise me on it and offer suggestions to fix it. That's what I call recognition and acceptance into a developer community - something I've never seen on LiveJournal.

This post wasn't made in an effort to change things. That type of change is up to people who have time, and want to put forth effort to change things. Considering I have about 3 projects I want to devote time to and 3 more projects that I think would be nifty to devote time to all above LiveJournal on my list, I don't think I'm one of those people. The post became more of a description of how things are rather than a how things should be - something I regret, but one of the side effects of writing about a project that has left me so bitter.

I do see eye to eye on that topic with you though - this post wasn't meant to motivate change. The change needs to come from someone other than me - I don't consider it my responsibility, or even something I'm interested in, to motivate the change here. I do admit that it's frustrating, but like I said, I'm concentrating on other things here.

I think that LiveJournal has gotten a big jump start with more paid developers. I think that you've been a primary force behind that, and I think that's pretty fucking cool, ya know? I just find it upsetting that all the little niggling things get left behind. That's not your fault, and it's not anyone else's - it's just the way it is. I'd like to see it change, but I'm not motivated enough to cause that change. I was at one point, but after ramming your head into a brick wall for a while, you just get tired of it. I got tired of it, and I turned away to projects that I enjoy, and that enjoy my help. I'm glad I did. I hope that LiveJournal can continue to be for you what it once was for me - a place to hack, and feel happy about it.

Good luck, Mark.

I agree wholeheartedly. I was involved a teensy bit with development a while back, and still read Zilla every once in a while, but looking at Zilla, it seems like things (important things) just fall by the wayside sometimes, and people that try to fix them are often met with apathy by the paid staff. I think that your bugs need to be fixed before you write whizbang new features, and while I understand the devs do what Brad tells them, to an extent, there has to be some priority somewhere for fixing the outstanding bugs. I didn't see this, so I stopped trying to learn how to contribute -- what's the point of patching if your patches will never be committed? Yeah, my translation patch was done quickly, but patches for more complicated things were taking forever to even be reviewed, much less checked in.

I don't want to see LiveJournal 'fail', and I don't think that not fixing the bugs will make it fail -- but I do think that not fixing the bugs is going to come back to bite them in the long run, and strongly feel that coding practices need to change.

(Deleted comment)
Now that's the nicest thing I've ever heard from you. Thanks, it means a lot coming from you. (And that's said in seriousness, not as a jab.)

I have moved on. My interest in LiveJournal development is not my main interest anymore. I'm already hacking on a number of other projects - projects where people actually respect the work I do and appreciate it. Dashboard (which I've written about) is one of those, as well as a number of other projects geared around RDF and so on. My time here was instructive, but it's not my main task anymore.

Mark was talking tonight about how Brad would like to hire someone else. I said that even up to a month ago, I would have applied for it - moving out to Portland and all. Now though, I just have no interest. I'd like to see LiveJournal improve - it is, after all, where I started in the open source world. But you're absolutely right in saying that this isn't the way to get respect. I've learned that, in the same way that I think you may have.

Thanks again for the compliment - and the advice, I'm already acting on.

Entries like this are informative to other people out there. I tend to get bitchy when I write about development, but I'm not really upset, just frustrated at the way things worked out. I still do think, though, that my assessments are fairly accurate. Plus, I'm way too meta not to post about LiveJournal in my LiveJournal - didn't you know?

I can't say I read all of this since it didn't interest me...

BUT I REMEMBER YOU PLAYING LUNAR ECLIPSE! Teehee. You'd leave. for like. a half hour. Then you'd come back like, "Oh, sorry! I was doing *insert random MUD-babble I don't understand*"


Actually the point of this comment was, "I was trying to think of the name of the MUD you played the other day, but never remembered to ask you and now I know."

But somehow the comment never made it there.

Whoa, dude, itching for a flamewar much?

/me blows kiss and wanders off. :)

  • 1