Definitions

Before we begin there are some definitions I have to set down.

Facebook Account

The actual Facebook account you use to talk to friends and family.

Facebook Page

A separate page you have setup for your brand or online community.

Facebook Application

An application you create which is basically just an set of IDs you need to connect to the Facebook API.

Setup

Before we begin you have to create your facebook account, page and application. I’ll assume you already have an account, or know how to create one. That’s clearly outside the scope of this tutorial. Make sure you remain logged into your Facebook account throughout this entire process.

If you don’t already have one, you need to create a Facebook page. Go to https://www.facebook.com/pages/create.php, and follow the instructions. When you are all done you will end up looking at your new page. It will have a URL that will have the following format.

https://www.facebook.com/pages/PAGE_NAME/PAGE_ID_NUMBER

Save the PAGE_NAME and PAGE_ID_NUMBER. If you already have a page, visit it and look at the URL for the name and/or id number. Make sure that your Facebook account has permission to post status updates on that page. I’ve only tested this on pages where my account is the page administrator. Your mileage may vary otherwise, but I see no reason it wouldn’t work as long as you have the necessary permissions.

The next step is to create a Facebook application. To do that go to https://www.facebook.com/developers/createapp.php and follow the instructions. You will go through a configuration process. You can leave all settings as the defaults. Just save the Application ID and Application Secret, and you will be good to go.

Get the Access Tokens Through OAuth

Now it’s time for the tricky parts. You have to give the application permission to access your account. Take the URL below and replace both instances of APP_ID with the ID of the application you just made.

https://www.facebook.com/dialog/oauth?client_id=APP_ID&
redirect_uri=https://www.facebook.com/apps/application.php?
id=APP_ID&scope=manage_pages,offline_access,publish_stream

Visit this URL in your browser and you will get a dialog box that will ask you whether to allow or deny granting permissions on your account to the application. The page needs to redirect somewhere after you click allow, and it will only allow you to redirect to a page your application owns. That’s why we put the URL of your application’s profile page as the redirect_uri.

The three permissions we are granting are manage_pages, offline_access, and publish_stream. Manage_pages grants the application access to pages which your account has access to. Offline_access means that the application will get an access token that never expires so we never have to do this complicated process ever again. Publish_stream allows the application to post new updates to your account, or any other place your account is able to post updates to, including the page we just created.

Click the allow button. Immediately go to the location bar of your browser and copy and paste the URL to a text editor. Do not lose it. You will see that the URL looks something like this.

https://www.facebook.com/apps/application.php?id=APP_ID
&code=CRAZY_LONG_CODE

That crazy long code is the magic we need. Take that code and use it to create yet another URL in the format that follows. Replace APP_ID with the Application ID as usual. Replace APP_SECRET with the Application Secret. Lastly, put that CRAZY_LONG_CODE where it belongs. Paste the resulting gigantic URL into the location bar of your browser and visit it.

https://graph.facebook.com/oauth/access_token?client_id=APP_ID
&redirect_uri=https://www.facebook.com/apps/application.php?
id=APP_ID&client_secret=APP_SECRET&code=CRAZY_LONG_CODE

If you did everything right you should now be looking at a big beautiful access_token. We will call this the account access token. It gives your facebook application permission to access your account. If you just want to post status updates to your personal account, you can take this access token to the next section. Otherwise, proceed to get the page access token.

In order to get the page access token, we need to visit yet another URL, it looks like this.

https://graph.facebook.com/USER_ID/accounts/?
access_token=ACCOUNT_ACCESS_TOKEN

The USER_ID is the user id of your Facebook account. Mine is apreche because my Facebook profile can be found at https://www.facebook.com/apreche. You might not have a profile URL, in which case you can use a numerical ID instead. If your personal profile is at a URL such as https://www.facebook.com/profile.php?id=NUMBERS then use the NUMBERS as your USER_ID. Take the account access token we got from the previous step and put it in place of ACCOUNT_ACCESS_TOKEN.

Now visit this URL in your browser. You are going to see some slightly complicated text that is in a format known as JSON. It should look something like this.

{
   "data": [
      {
         "name": "PAGE_NAME",
         "access_token": "PAGE_ACCESS_TOKEN",
         "category": "Website",
         "id": "PAGE_ID"
      },
      {
         "name": "APPLICATION_NAME",
         "access_token": "APPLICATION_ACCESS_TOKEN",
         "category": "Application",
         "id": "APPLICATION_ID"
      }
   ]
}

Depending on how many different pages and applications you have on your Facebook account, you may see more than this. Regardless, you are only interested in one thing. Find the section with the PAGE_NAME of the page you want to post to and extract the PAGE_ACCESS_TOKEN from that section. This is the token we have been looking for. Save it and keep it safe.

Making the Post

Now that we have the page access token, we can finally do the actual status update. I will use curl in my examples since that is language agnostic. All you really need to do is make an HTTP POST. In Python you could use urllib. You could also use an appropriate Facebook library for your language, such as the Facebook PHP SDK. It’s up to you to learn how to get this done in your programming language of choice.

Here is what a complete POST looks like using curl in the shell. Remember to put the PAGE_NAME and PAGE_ACCESS_TOKEN where they belong.

$ curl -F access_token="PAGE_ACCESS_TOKEN" \
-F message="testing" \
-F picture="http://i.imgur.com/2uLkT.jpg" \
-F link="http://frontrowcrew.com" \
-F name="Front Row Crew.com" \
-F caption="FRC In the house!" \
-F description="Homepage of GeekNights and other fine free entertainment." \
-F source="http://www.youtube.com/e/NKWpGJ4Xhw8?autoplay=1" \
https://graph.facebook.com/PAGE_NAME/feed

You don’t actually have to include all of that information with every post. Just include the information that is relevant. I have simply included all of the possible information to demonstrate a complete example. Here is what the post above will look like on your Facebook page.

The avatar and the username on the posting are going to be the avatar and username of the page itself. You can’t change those from post to post.

The word “testing” that you see there is the message of the post. Most of the time I imagine you are going to be posting just messages without any other information.

After that is a separate section with the link, picture, video, and everything else. Depending on which combination of information you post, that section will work differently. You only get one of these sections per post, so depending on what you want to do, you might need to stretch it out over multiple postings. If your post only contains a message, this section won’t exist at all.

The link field controls the URL where the user will be sent if they click on the blue text at the top of the section. The name field controls the blue text itself. If you don’t specify a name then Facebook will visit the link and figure out a name to use. In this example the link is http://frontrowcrew.com and the name is “Front Row Crew.com”. If you know HTML you can think of it like a basic “A” tag, since that is what it is.

<a href="LINK_GOES_HERE">NAME_GOES_HERE</a>

The caption controls that light gray text directly beneath the link. The description controls the block of text that is one space beneath the caption. If you leave either of these blank then Facebook will visit the link in question and try to figure out a caption and description on its own.

The picture is where things start to get a little bit complicated. First of all, if you specify a link without a picture, then Facebook will actually pick an appropriate picture based on the link, just like it does with the name, caption, and description. Since it will probably pick a bad picture, I suggest you set the picture manually whenever specifying a link. The picture is just a link to any image on the web. Clicking on the picture thumbnail will take the user to the URL of the link.

You can actually specify a picture all on its own without any link at all. In that case clicking on the picture will take the user to see the picture in its full size. The caption, and description will be extremely barebones unless you manually specify them.

Source may as well be called video instead of source. It needs to be a URL directly linking to a video file, usually a flash video. I’m pretty confident other HTML5 video formats will work, but I haven’t tested them myself. Don’t make the mistake of setting the source to a video’s page like this.

http://www.youtube.com/watch?v=NKWpGJ4Xhw8

That won’t work. You have to set the source to be the actual video file as in the example.

http://www.youtube.com/e/NKWpGJ4Xhw8?autoplay=1

You are on your own on how to structure these links on other video sites besides YouTube. I think I can safely assume the vast majority of people are using YouTube.

If a source is set without a picture, you may get an error if Facebook can’t figure out a video thumbnail on its own. In this case, you must specify a picture to successfully make the post. The picture will be used as the thumbnail. Clicking on the picture will cause the video to play. For YouTube the player is embedded within Facebook itself. As always, your mileage may vary with other video formats and sites.

Now, Facebook is actually pretty smart. Let’s say you specify just a link, and that link goes directly to a YouTube page. Facebook will automatically figure out the source and picture on its own, assuming you do not manually override them.

Lastly, what if you specify absolutely everything as in the example above? Obviously you will have complete manual control over all of the text appearing in the post as well as the picture. Clicking on the picture will play the source video. Clicking the blue text link will always go to the specified link. In this way you can actually link to two different things. Clicking on the picture will play a video, but clicking the link can take people to your blogpost about the video or other related place on the web.

Also, it should be noted that there is nothing forcing the picture and video to match. You can post a thumbnail for a video that is not related to the video at all. Users might be very confused if you abuse it, but there are legitimate reasons for doing so. You might just a brand logo as a thumbnail on a video which does not include that logo.

One last thing. The post will always return some information formatted in JSON. If it fails there will be an error for you to read. If it success there will be an ID that is the ID of the status update you just posted. You can save and use that ID for later if you want to edit or delete the post.

Conclusion

Figuring this out cost me many hours. If this saves even one person from that struggle, then it was worth it.

I hope this can also serve as an example to other people who write technical tutorials or howtos. You can’t just skip over details. You need to include every step and explain the what, why, and how of all those steps. Otherwise readers have to go to ten different sites to piece together the complete picture like I did. Even worse, users might blindly copy and paste without understanding what they are doing.

And of course, I hope this shows just how much of a pain OAuth can be when it is used unnecessarily. With my suggestion of every account also being an application, many of the OAuth steps would be eliminated without sacrificing any security or privacy.

But now we run into problems of security. What if someone else calls the lab impersonating your doctor and gets your test results? What if you change doctors and the lab releases the test results to your old doctor against your wishes? What if the lab is full of jerks and they send all sorts of crap to your doctor in your name. We have this exact same problem on the Internet, and that is why OAuth was created.

For decades the most popular means of authentication on computers has been usernames and password combinations. You want to prove to some system that you are a certain person, so you enter in a secret piece of information that only you know to prove you are you. Now that you have identified yourself the system will allow you to engage in activities permitted to your account.

Now let’s say you have a newsletter system, and you want it to send out emails on your behalf. Even to this day a very popular way of doing this is to give the newsletter system the username and password to your email account. The newsletter system will authenticate on your behalf, so your email server will think that the newsletter system is you. That’s what authentication is all about. As far as your email server is concerned the newsletter system is you, and has permission to do everything you do, including changing your password or deleting your account.

It’s obvious that this is a problem. Even though that goes against everything you’ve ever learned about security and passwords, people still do things this way. You should never tell your password to any other person ever, not even your closest and most trusted people like spouses. You definitely shouldn’t tell it to other computer systems. You should only ever send a password to the system for which it is intended. Your email password is for you, your email server, and that’s it.

Well, I made this fun game on the web, and my users want to tweet out their high scores. I could just suggest they do it with copy/paste, but that’s like driving back and forth across town to get the test results. Instead, I want my game to do the tweeting on behalf of the users for the greatest efficiency. Now I could, and many systems did, just ask for the users to give me their twitter usernames and passwords. It is possible I could be honest and not abuse that, but do you trust me? Even if I was an entirely client-side application, and your password is being sent directly to Twitter, this game is not officially endorsed by Twitter. How do you know I’m not secretly sending your password to my own servers or doing something else shady?

Thankfully we have solved this problem, and the solution is OAuth. OAuth is slightly complicated, but it gets the job done. Let me explain it as simply as I can using our high score game tweet example. We’ll call the game shootyguy.

You have a Twitter account, and I want you to let shootyguy send tweets on that account. First shootyguy gets its own special twitter application account. That account has an ID and a secret, which is basically the same as a username and password. Now shootyguy can authenticate with Twitter and prove that it is indeed shootyguy.

When you get a high score for the first time it will give you a link saying “click here to let me send tweets for you!” That link is a link to Twitter. It contains shootyguy’s ID number and a list of the things shootyguy wants you to let it do. In this case the game is only asking permission to send tweets, but it could also ask permission to edit your avatar, change your profile, etc.

You click on this link and you are sent to Twitter itself. It’s safe to give your Twitter password to Twitter, obviously. You do so, and Twitter asks you, “Hey, do you want to let shootyguy do the following things with your account?” If you say no, nothing will happen. If you say yes, then Twitter will remember that shootyguy has permission to send tweets for you. It will also send shootyguy a special secret password that it can use to act on your behalf. If shootyguy is a popular game, it’s going to have to remember those secret passwords for every user who has tweeting enabled. As long as the game holds onto these secrets, and you don’t take permission away from it, it won’t have to ask permission again. This painful user interface problem of jumping back and forth between web sites only has to happen once ever.

This is what is called authorization. You see, both you and shootyguy authenticate with Twitter to prove you are who you say you are. You are authorized to do whatever you want with your own account. Shootyguy is authorized to send tweets, but nothing else, on a whole bunch of accounts that have granted it permission. Up until relatively recently we didn’t have authorization, just authentication. Now that we have it, it’s easy to see why it is so incredibly important.

Perhaps the most important feature of authorization is the ability to deauthorize. Imagine if shootyguy got hacked like the PSN and it had a database full of Twitter passwords. That would be extremely bad. You would have to change your password before hackers started using it. If they had used OAuth, you could just remove authorization for the compromised application, and your account would be perfectly safe. Worst case the hacker starts doing the few things you authorized them to do before you turn them off.

I think I have made it very clear that in cases where you want to give a third party access to one of your accounts, OAuth, or a similar authorization system, is an absolute necessity. Anything less is grossly negligent.

Given that, what is up with the title of this blog post? Shouldn’t it be “Always OAuth”? Well, that seems to be the opinion of many sites out there such as Twitter and Facebook who are forcing OAuth as the only way to interact with their APIs. I can see why they do this. OAuth is more complex to implement, so developers would keep taking the easy way out by collecting passwords if they were not forced to use OAuth. It’s also next to impossible to educate users on not giving out their passwords.

Yet, there are still those rare occasions when OAuth becomes a huge pain in the ass for no benefit whatsoever. Let’s turn the tables around a little bit. Let’s say that we make an official shootyguy twitter account. We want to automatically tweet on that account whenever a player beats the game at shootyguy.com. How does that work?

In this case, we don’t need to ask the player for authorization. We aren’t using the player’s Twitter account. Shootyguy is using its own account. There’s no benefit to using OAuth in this case. It would just be easier for shootyguy to authenticate with its own username and password, and have full authorization to tweet with its own account.

The problem is that shootyguy is a program, not a person. It can’t just visit twitter.com and send a tweet. It has to use the API to tweet programmaticaly. But the API forces you to use OAuth no matter what. Since it is a program, and not a person, how can it click the “Allow” button in the browser? It can’t. What you have to do is manually do the OAuth in a browser and collect and save all the secret keys along the way. Then you put all the secret keys on the shootyguy.com server so it can send tweets.

Is it really that hard? No, there are many things that are more difficult, but it’s still a pain. In a way it makes sense. The shootyguy account is giving permission to the shootyguy.com application to tweet on its behalf. But Twitter is actually pretty simple as far as OAuth goes. If you want to see frustration, try it with Facebook.

Pretend that instead of posting to a shootyguy twitter account, we want to post onto the wall of the official shootyguy Facebook page. Well, a Facebook page isn’t a Facebook account, or is it? And a Facebook account isn’t a Facebook app. I’m going to write a tutorial on every step involved in doing this, but here’s the TL;DR version. You have to create a facebook account, a page, and an application. You have to authorize the account to post onto the page. Then you have to authorize the application to have a bunch of permissions on the account. Then the application has to use its secret keys on the account to get yet another secret key it can use to access the page.

Again, this is actually a great system when you are using it for what OAuth was designed for. If you want to give the game permission to post on your personal Facebook wall, this is fantastic. For the game to post on its own Facebook wall this is awful.

Of course, my complaining isn’t for naught. I actually have a solution that doesn’t require changing OAuth itself in any way. The solution is actually quite simple, and it actually opens the door to more features for users that have been unexplored.

I won’t hold back, the answer is to make every account also an application that has full authorization of itself. Why are applications and accounts separate entities? Why do I have to make my own Facebook application and connect it to my account? If my account were also an application, it would only need authorization to do things with other accounts, but not as itself. I shouldn’t have to pretend to be an authorized third party application to post on my own personal Facebook wall through the API.

One side benefit of this is that every user account can be authorized to perform actions on other user accounts. That can be dangerous if people give too much authorization away, but I think it will be very clear to people what is going on. “Do you want to give your friend joeyjoejoe1337 the ability to edit your profile? Yes or No?” It’s not a question too many people are going to get wrong.

While you may not immediately realize it, there are many cases where people will want to say yes. A parent can give their children their own personal private accounts, but can then authorize their own accounts to have a certain amount of access. You could authorize your trusted friends to handle your accounts in case of emergency without giving anyone your passwords. You could have a company account and give many people access to it without sharing a password around the office. You could then easily add and remove authorization as employees come and go. There are many duct tape third party systems that add this functionality for businesses that could be done away with completely if my idea were implemented.

I hope from reading this that you have a good conceptual understanding of the difference between authentication and authorization. I also hope you understand what OAuth is, why we have it, and why it is so great. And though I know it won’t change anything, I just wanted to complain about the unnecessary complexity that can make OAuth sucky under certain circumstances. If you are building an application, make sure you use it appropriately, and please try out my idea of every account being an application. And remember, never ever give your password for anything to absolutely anyone at any time under any circumstance. It is never necessary. No, not even in that circumstance you just suggested.

Firstly, these attacks are not really all that effective. Defacing web sites? Releasing customer data? That doesn’t make any difference in the grand scheme. Do you remember viruses like the ogre? It erased all drives in a computer. When was the last time someone wrote a virus that actually did serious damage like that? I’m not aware of any in recent memory.

Why don’t we see such harmful attacks anymore? Is it because that sort of attack doesn’t serve the motivations of the hackers? Is it because our modern systems are properly hardened against these kinds of attacks? Are these attacks happening and not being publicized?

Whatever the reason is, it is both disappointing and encouraging. You see, I can respect this sort of attack in the way that I can respect someone like Mike Tyson. His boxing is a sight to behold, but I have no desire to behold a man having his ear bitten off. You can impress me by writing a virus that turns off CPU fans and forces people to buy new PCs, but I don’t actually want to see that to happen. That makes the world a worse place by wasting a bunch of valuable resources for no benefit.

Secondly, these attacks lack technological sophistication. SQL injection and DDoS against soft targets? Please. These techniques aren’t demonstrative of deep technological knowledge. Regardless of my feelings on the motivations or target of an attack, as a technology professional I have a hard time respecting such easy hacks.

Imagine someone who burglarizes an old grandma who left the door unlocked. Even if it was a mean old grandma who needed to be taught a lesson about security, that thief isn’t going to get love from anyone. Because the burglary was so easy, the perpetrator is easily branded as a coward and lowlife scum.

Meanwhile, there are two other kinds of thieves who do get love and respect that makes up for their evil ways. The first is the professional thief or con-man. Their talents are so great that we can romanticize their stories and root for them. Despite acting immorally, their biographies become hit movies. The other kind is the Robin Hood who commits a crime for a good purpose. They do something that is illegal, but is not viewed as wrong given the circumstances.

What I want to see happen are some attacks that are both technologically difficult and have effects that benefit the world in some great way. If you are willing to so blatantly ignore the law, unlike myself, then cut it out with the lame hacks. Do something big and serious that in and of itself will have a huge positive effect for all people on the planet. Don’t be that guy who robs a convenience store at gunpoint. Instead, strive to be like Lupin III or Frank Abagnale.

Since things like this haven’t really been done before, I think I have to give some examples. Here are my top ten ideas of possible hacks that would have a huge positive benefit on society. The perpetrators of these attacks would definitely earn a great deal of respect to go with their time in prison. Post your own ideas in the comments!

1. Hack stores like iTunes or Steam to give everyone free games, movies, music, etc. Remove region locking on services like Hulu and Spotify. Allow art and culture to be spread around the world for free.
2. Release the source code to important programs like Adobe Creative Suite, Microsoft Windows, iOS, the Google Search algorithm, etc.
3. Take control of ISPs to remove filtering, bandwidth limiting, wire tapping, and other nefarious systems.
4. Hack into the systems that host and sell academic journal articles, and get the full text of all the articles onto the net somewhere for free.
5. Hack into the world’s financial systems and bring down all the evil investment banks like JP Morgan, Goldman Sachs, etc.
6. Hack into government and corporate back office systems to expose corruption around the world. I’m sure there are plenty of Enrons waiting to be discovered.
7. Completely disable industrial facilities that are hazardous to the environment. Even if they come back online, a few days without something like coal burning makes a big difference even at the expense of a power outage.
8. Completely bring down the great firewall of China or other Internet censorship systems.
9. Attack the television satellites and turn off almost every TV in the world for a significant time period. At the very least replace Fox News with Al-Jazeera English.
10. Disable the nuclear arsenals of every country that has them. Let the power plants stay, just disable all the missiles.

I think you get the idea now. LulzSec and others have been thinking too far inside the box. The lack of imagination and lack of technological skill is why nothing like the attacks I’ve describe has happened, or will happen anytime soon. If anything like the items on my list even comes close to happening, I will be more impressed than I have been in my entire life. As of right now, I’m still yawning.

I fully expect to wake up tomorrow to see that my computers will not boot, and they just display laughing skulls all over the screen. I’d be mad that I would have to fix all my computers, but a little bit happy that that can still happen.

Just about every day I read about some article about turning New York City into Silicon Valley 2. What bothers me is they take it for granted that this is a thing that is a good idea, and move right along to discussing how it can be done. It’s obvious why the startup people want to do this. There are a lot of people and, more importantly, a lot of dollars in NYC. It’s the most important city in the world. Yet, there are few technology-centric businesses based here. To these people the city is a huge untapped mine filled with gold.

The thing is, the location of technology companies doesn’t really matter that much does it? If I were starting a tech company I would probably pick Kansas City once Google brings the fiber there. The Netherlands is also a good choice, and has the fiber right now. The only real reason they come to NY is because they need more developers. The valley is tapped.

If they have marketed these entrepreneurial ideas so well, why are they having a hard time getting these developers? Why can’t they replicate the valley in New York? What is giving them all this trouble? I’ll make a general trollish statement and say that New Yorkers aren’t morons like those Californians are. Less trollish statement, New Yorkers aren’t as naive as Californians.

You see anyone who has a silicon valley attitude already probably moved there, or wants to move there. If someone stayed in New York it’s because they have a New York attitude. We aren’t the kind to work all night eating only instant noodles in exchange for a miniscule chance of getting rich and making some venture capitalist much richer. We have something here called rent, and it’s really expensive. We also like to eat good food, which is also very expensive. If you want us to work for you you have to pay us in cash, and lots of it. Venture capitalists don’t like that. They want kids right out of college who will work for peanuts because they don’t know any better.

Also, while New York lacks technology companies, it doesn’t lack technologists. Every desk has a computer on it. There are thousands of technologists in this city, many of them among the best in the world. The thing is, we all work for companies that do something else besides technology. Many work for financials that make money by having money to begin with. Some work for insurance companies, media companies, advertising firms, or *gasp* companies that actually have a product to sell. In other words, we work for businesses that actually make a real profit with a real sustainable business model that hasn’t been made obsolete like Hollywood has.

All these venture funded startups, how many make real profits? LinkedIn just had a huge IPO, how much profit do they actually make? Apparently  the company is valued at 521 times its profits. I don’t know how math works in CA, but in NY we know how to use a calculator. That doesn’t add up. If we’re going to work somewhere and get paid, it better be someplace that actually makes money. If you promise me the salary I demand, you better damn well not bounce that check.

These people are treating investment dollars like they’re revenues. In NY people are still living in the real world. We’ll get excited about real things, like sales figures going way up. If some morons decide to invest a huge sum in a company that isn’t turning a profit, that’s not something to cheer about. It’s something to be very worried about. If I see the champagne opening, my first instinct is to run.

I also have a suspicion that the weather makes a big difference. New York has nice weather, but the winter is cold and snowy. That has a huge psychological effect on people. Because we are often grumpy, we want to escape. If you make a New Yorker rich, you know what they’ll do? They’ll buy a luxury apartment, a house in the Hamptons, and a tropical beach house. Then they’ll retire immediately never to be seen again. Make a valley entrepreneur rich, and they’ll show up to work the next day like nothing happened. The weather is so nice, they are happy with their late nights and ramen dinners. They don’t want to escape, they are happy to work forever.

If you’re one of those people trying to push your silicon valley mentalities in New York, let me be the first to tell you to literally get out of town. You’re not wanted here. If anything, I think you guys should take a little New York back home with you. When you have real money and are ready to pay my rent, then give me a call. Until then, save your cash for plane tickets.

Despite the greatness in the indie video game scene, there is also a great deal of fail. Because indie developers lack the resources of giant corporations like Nintendo or Electronic Arts, there are certain flaws in their games that we have to accept. The controls won’t be as polished and smooth. There might be graphical glitches on certain video cards. The game might crash under weird circumstances. Without the money or time for thorough testing, this is just a reality of life we must accept. I forgive the indie games for these kinds of flaws.

That being said, there is one area in which failure is absolutely not acceptable. That area is online multiplayer. It seems that just about every week a cool new multiplayer game comes out on Steam, but the networking is a complete disaster. This is absolutely unacceptable. If it’s primarily a single player game with a small online component, then it’s no big deal if that part doesn’t work. However, if it’s primarily an online game then the game may as well not exist if the networking is busted.

As a programmer I know full well how difficult it is to write netcode that works. There are so many factors to balance including latency, synchronization, cheating, firewalls, etc. It’s no doubt a huge pain to write good net code. To those that have succeeded in this task, I salute you.

Even worse, writing netcode is a distraction from making your game. A game developer wants to make their game as good as possible. In a competitive online multiplayer game that means working on things like balance, controls, user interface, level design, and adding more modes of play. Every minute spent working on netcode is a minute a developer is distracted from the game itself. It’s no wonder that developers don’t spend much time on it.

That being said, if your game is primarily an online multiplayer game, it may as well not even exist if the netcode doesn’t work. I acknowledge that it is difficult code to get right, and that it distracts from the task of actually making the game. I just don’t accept those as valid excuses. Networking is hard, not impossible.

In 1996 Id software released QuakeWorld. The original Quake had netcode that was only suited for LAN play. QuakeWorld fixed it so that Quake could be played beautifully over the Internet. Tribes 2 was released in 2001. The netcode was so amazing that the game was playable with a 56k modem. We are living in the year 2011. No matter how hard the netcode problem is, it’s been a solved problem for fifteen years. I’m sorry. There is absolutely no excuse for getting it wrong.

One reason people fail is because they try to do something other than the classic dedicated server model. In my opinion, this is the way all non-MMO games should do online play. It’s a design that works beautifully for over a hundred thousand Counter-Strike players every second of every day. I’m all for trying new things. Rotary engines are pretty cool because they actually work. If you want to try to make a new kind of network engine, that’s great, but it had better work.

Not following this advice is how we get games like Frozen Synapse trying to use centralized servers that they can’t keep available. It’s a turn-based game, and they can’t even keep the server up. We also get games like Ace of Spades which wants to launch the game from in-browser links instead of typing IP addresses into the game itself. Most times when you try to play you just get a connection error. If these developers had done things the old fashioned way, they might be in better shape.

Another mistake developers make is they don’t build the networking in from the ground up. Having online play affects the overall design of your game in many ways. If you don’t code your game with networking in mind from the very beginning, it will be enormously difficult to properly add it later on.

Minecraft has this problem, and that’s why so many people keep making all sorts of server mods for it. Even with the mods I consider the Minecraft multiplayer too awful to be even worth playing. Notch has the money now. If he wants multiplayer to not suck, he should build a new Minecraft that is designed for networking from the start. They can call it Multicraft.

Civilization V is a massive failure in this regard. Not only are their lag and connection issues, but the actual game mechanics actually don’t hold up in multiplayer. The simultaneous turns give a huge advantage to the first mover in combat. When Civ IV came out, people stopped playing Civ III. The fact that many people are still playing Civ IV after Civ V has been out for months shows you they really messed something up.

Magicka is a game with a ton of network problems. Even after tons of patches, the game still gets out of sync. Diablo II was perfect in 2001, so why is Magicka flawed? The Torchlight guys have the right idea. They have no networking whatsoever in Torchlight. If you can’t do it right, don’t do it at all. They are making Torchlight II with networking in mind from the start. I have high hopes for them doing a good job.

Natural Selection is my second favorite FPS of all time after Tribes 2. It’s a Half-Life mod, so the networking just works perfectly. I was very excited for Natural Selection 2, and I pre-ordered it immediately. The developers started out on the Source engine, but abandoned it to write their own. While their engine is very impressive, they have failed at networking. Their experience comes from making mods, so they had no experience with networking. Perfect netcode was provided for them by the underlying engine. Natural Selection 2 is still in beta, but the networking is awful. There is insane lag. It gets better with every patch, but it is still nowhere near satisfactory. At least we’ll always have NS1.

You might argue that some of these games are commercially successful despite their sub-par networking. That is true, but they are very lucky. Introversion software released a great RTS called Multiwinia. When it was first released many people could not connect to any servers due to network errors. I don’t know if they eventually fixed it because I stopped playing. The broken online play prevented a community from forming around the game, and the game died. Compare that to Defcon, another Introversion game that has working online play, which still has a community to this day.

I’m sure everyone is also familiar with the Nintendo way of failing at online multiplayer. Friend codes and other weird systems create a huge barrier to online play, even if the netcode works. Borderlands has fine netcode, but reliance on GameSpy accounts makes it annoying as hell to get it running. Steam and XBox Live are both perfect. Every single game should integrate as much as possible with one or both of those systems. Why would you intentionally create a barrier to entry for your game?

Networking is hard, so it is obvious why so many game developers fail at it. It’s just incredibly frustrating for us as gamers to spend money on a multiplayer game that has broken online play, where there is really no excuse. Almost every game should use the standard model of dedicated servers integrated with XBox Live and/or Steam as fully as possible. It’s bad enough that so many games with big potential die out from lack of interest, it’s an even greater shame if it’s due to technological failure.

Right now I’m at a point where I buy all kinds of cool indie games only to be disappointed when the networking doesn’t function. I’m at the point where I’m not even going to buy games now until I have confirmation that they work. With more technological failures more people might take on this attitude, and it could be a serious blow to indie game developers, and thus a serious blow to video game innovation. Nobody wants to see that, so just make the networking work. There’s no valid excuse.

Take for example the Nintendo DS vs. something like the Pandora. The Pandora is really no comparison. Just looking at it makes me shudder it’s so hideous. There’s little incentive to actually develop anything for it since nobody owns one. Hooray, you made a game for the Pandora that nobody will ever play.

Meanwhile, the Nintendo DS is a closed system. Nintendo will only give you a dev kit if they approve you, and you pay them money. Then you can only release your game if they approve of that. This hasn’t stopped people from reverse engineering it and homebrewing the heck out of it. There is more homebrew action going on the DS than on even the most popular open systems out there. The incentives of developing for such a widely used well designed system make with worth the hassle of hacking and reverse engineering it.

The same is true for almost all consumer electronics. We have a bunch of open devices that suck and don’t really work, and we have awesome closed devices that we have a hard time pushing beyond their limitations. What would happen if one of the companies making closed devices would just cave and open it up?

Actually, this has happened. Do you remember the Linksys WRT54G router? I don’t have sales figures, but it’s obviously one of, if not the, best selling routers of all time. Because of the GPL, Cisco was forced to open the source code of the firmware for early versions of the router. The result is that a huge community sprung up of people buying this router and flashing it with firmware that added a ton more features. This obviously helped their sales by a ton, so why did they change the software in later versions and keep their future routers closed? If they had kept their routers open, Netgear, D-Link, and everyone else would have no chance of competing unless they opened up as well.

Now, depending on the device, going completely open can pose problems. Sometimes it can allow software piracy, like with the iPhone or DS. Sometimes it can allow media piracy, such as with an open source Blu-Ray player. But even in those cases, an official dev kit that anyone can buy will not help piracy that much.

Even with completely closed Blu-Ray players, the whole system is already cracked wide open. Pirates are pirating, and will continue to pirate. Little to nothing can be done to change the rate of piracy. Yet, imagine if Sony made one particular model of Blu-Ray player that was open. Without marketing it heavily, they just had a page on their site with documentation and development tools for the player. Maybe they even discreetly sell a dev kit for a token fee. That will almost immediately become the #1 Blu-Ray player, and no other will be able compete with it.

Imagine if Nintendo discreetly sold official DS flash cartridges with a development kit. They would make a ton of money since all those dollars spent on R4 cards would be in Nintendo’s wallet. Piracy of DS games is rampant and unstoppable anyway, this would at least give more money to Nintendo instead of the people who make the R4.

Of course, this will never happen. These big old companies are set in their ways, and there is pretty much zero chance they will be smart enough to do something like this. They are too stubborn and old fashioned.

That doesn’t mean it can’t happen. I’m calling out all the people who make open hardware to change their ways. Instead of making something like Pandora, make a product with a consumer focus. If you design for developers, you’ll only get developers buying it. Be like a real company and focus completely on consumer sales. Just while you’re at it, make the device open and make development tools available or for sale. Don’t heavily promote the openness, promote the device itself. By keeping the openness quiet you won’t scare anyone away, and you’ll grow a sleeper hit from the underground. Think of it as Linksys router on purpose.

Imagine yourself right now trying to buy something like, oh, a television. There are so many to choose from in different sizes with different specs. But one feature that no television has is openness. Imagine if there was just one model of television that had a dev kit available on some page of the manufacturer’s web site. They didn’t promote it or draw attention to it, but they had it there. Would that not immediately become the most popular television? Of course it would. It would be the obvious choice of almost every nerd. After the nerds got done with it, it would become the obvious choice of non-nerds as well.

If you are a consumer electronics manufacturer, please make your stuff open. It’s in the best interest of your company and your customers. Throw away your old ways of thinking. Just do it. Just try it on one model. If it doesn’t work, cancel it. At least try. What’s the worst that can happen? It sells the same as it would have sold if it were closed? That’s not likely.

My first experience with Linux was in the very late ’90s. The first time I saw it, some kids at the nerd summer camp I went to were running it. I didn’t use it, and I didn’t learn it. Yet, as with all things related to computers, I was curious. In high school a friend lent me a Corel Linux CD-ROM. I tried with no avail to make it work on the family computer (486 100mhz). I didn’t realize I had to boot from the CD-ROM, and I don’t think that computer even had that capability.

It wasn’t until 1999 that I downloaded Red Hat ISOs over my 56k dial-up connection. It was the first computer I ever built and personally owned, a Pentium /// 450mhz. I was able to dual boot Red Hat 6.0 and Windows 98SE. The thing is, I soon deleted it. It didn’t work with my USB dial-up modem. Linux was apparently just a different desktop with some different looking applications, and bad hardware support.

It wasn’t until I got to college that I actually learned something. All the computer science labs ran Solaris, and I started to learn UNIX for real. I soon discovered Mandrake Linux. It actually worked with all my hardware, even my weird PCI IDE controller. Most importantly, I learned to SSH into the Solaris systems to do my school work, and even use X11-forwarding.

At this point I was hooked. I was running Linux as the primary OS in a dual boot system. I only loaded up Windows (2000) for PC gaming. The thing is, hardware support was still a problem. My hardware didn’t all work perfectly, and Mandrake didn’t update quickly enough to give me the updates that fixed those problems. That’s when I had the distro rodeo and picked Gentoo.

I tried every major Linux distro at the time, and BSD as well. None of them really impressed me, and Gentoo didn’t even seem to work. Yet, I kept going back to it. The splash screen was so good looking, the community forums were so helpful, and the documentation was so great, that I kept at it. Eventually I came to realize that yes, I actually had installed it properly multiple times over, but unlike other Linux distros X was not a default part of the system. Booting into a command line wasn’t a failure, it was success.

From then on I was Gentoo crazy. I would constantly be rebuilding packages updating packages, even reinstalling the whole system. I would constantly be tweaking and twisting system settings to see what they did. Mostly I messed around with the user interface. I even fell into using fvwm, the most customizable X window manager. By default it sucked, but if you configured it, you could go beyond anything else in existence. Boy, did I ever configure it.

Late in my college years, and after graduation, I was soon employed. My free time evaporated. I was spending a lot of time commuting to work. My iPod became a lot more important than my desktop. I needed Windows to run iTunes, and to play games. I had little desire to code outside of work. Yet, I still kept a dual booted Linux because it was nicer to SSH into my blog/podcast server from that, than from PuTTY.

By then, I was rocking Ubuntu. It just worked. It installed in 15 minutes, not three days. I didn’t need to jump through any hoops to get my NVidia card working. I no longer needed to edit the X configuration to get dual monitor support and proper resolutions. I still had a natural inclination to customize the user interface, but it was kept at a minimum simply due to the fact I had no free time for it.

Eventually I was running Ubuntu and Windows. Then my desktop was dual booted, but my laptop was just Ubuntu. And then when I discovered VirtualBox, and it became good enough, I ran Ubuntu on the desktop in a virtual machine, and only had Windows installed on the bare metal. And just recently I replaced that laptop with a new one. This new laptop is so powerful that it too can run Linux in a virtual machine. And thus, my days of installing Linux are over. I wish I could run Linux in the bare metal and Windows in a virtual machine. The thing is that I use Windows for gaming, and VMs are not so great for that. I use Linux for web development, and running it in a VM makes no difference for that at all.

You see, in college I had a ton of free time. I spent that free time working on my computer. I do not regret spending my time in that manner. Most of the Linux skills I use every single day I learned from rocking Gentoo. Even so, I can now look at my old self and laugh. I was not unlike a mechanic who always worked in the garage and never drove anywhere. I spent all that time trying to customize the computer to my preferences, that I hardly ever computed. That’s why I don’t have my own software business. While I was compiling and configuring other people’s code, the other guys were writing that code.

Thankfully I learned my lesson, and I hope you can learn the same. It is often quite difficult to change computers. You can spend hours just customizing keyboard shortcuts, let alone other settings. Then, as soon as you use a different computer, all of your customizations are gone. Even if your customization can increase your efficiency, is it really worth it if it takes you an hour to set it up?

Take it from someone who has been there. Instead of changing the computer, change yourself. It’s easy to change yourself, you have 100% full control. You don’t even need to spend time looking up how to do it. Get used to the default keyboard shortcuts instead of making your own. Learn to live with the default settings. Once you do, you can be just as efficient as you would have been with your customizations. Better still, you will be able to sit at any computer and get going immediately without being frustrated at a different setup.

It used to be that my computer felt like home, and every other computer felt like a foreign country. Thanks to cloud computing, any computer with an Internet connection and a web browser can become your home as quickly as you can login. You no longer have to spend a day installing and configuring software after you have a fresh OS install. You just have to install your favorite browser, and maybe one or two other things like Steam or iTunes.

Learning default settings only makes this even easier. I truly feel at home on absolutely any computer I sit in front of. I am familiar with all three major OSes and all the major browsers. I still do not prefer OSX, which I am forced to use at work, but I know it. I’ve also made vim my text editor of choice, and I use it with a very default configuration. Vim or vi is already installed on almost every system, I can easily use any server I come across. People who are perhaps used to graphical editors, like Eclipse, may have some slight difficulty if they have to SSH into a server and fix something.

If you get frustrated behind the wheel of any car other than your own, then do you really know how to drive? A real driver can sit in any driver’s seat and be off to the races after a quick seat and mirror adjustment. That’s not to say you should never spend time under the hood. It’s very important to know how things work, and to be able to fix them when they go wrong. Just don’t spend so much time under there that you never leave your garage.

You won’t get very far in this world by working on your computer. If you want to make it somewhere, you have to use the computer to compute. Start now.

It dawned on me that because of HTML5, it should actually be possible to create an HTML presentation with video that is seamless. Today I went ahead and prepared a video-centric panel for Connecticon, and I did it all in HTML5. I realized afterwards that I should build this into a tool, so that others can use it as well. Hence, Presentoh was born.

http://github.com/Apreche/Presentoh

The idea is really simple. There are only four kinds of slides I think you should be using: title slides, bulleted lists, videos, and images. I built a simple HTML template which could handle all four types of slides. I made a brain-dead simple CSS style for it which entails white text on black background. The black background also looks more professional because photos and videos really stand out. Also, it looks better when using a projector in a dark room because you can’t see the borders of the projection.

The only catch was that I wanted to use my Logitech Cordless Presenter. I went out and found jquery.hotkeys, which allowed me to bind any key on the keyboard to move between slides by redirecting in JavaScript. Problem solved.

All you have to do to use Presentoh is create a file containing JSON which defines all the slides in your presentation. Then you just run the presentoh.py script, tell it which json file you want to use, and it spits out all the HTML files for your presentation. Find the first slide in your presentation, open it in your browser, go full screen, and rock and roll.

I think it’s pretty awesome that while this tool is really hacky, and doesn’t have any error checking, it still beats out Google Docs, Powerpoint, Open Office, and all the rest in terms of how it handles video. There are some slight catches, but nothing that can’t be worked around. For example, you have to make sure all of your videos are the right codec for the browser you are using. My H.264 videos obviously didn’t work in Firefox.

Also another issue is that there is no way to automatically make an HTML5 video go full screen. If you’re wondering why, the answer is on stackoverflow. Therefore you have to manually specify the height and/or width of each video depending on its aspect ratio, and the resolution you will be presenting in. This is to prevent videos from going off the edges of the screen if they are too wide or too tall, and to prevent aspect ratios from being broken.

All the rest of the information can be found in the README file. I hope Presentoh helps somebody out there with their presentation needs. Enjoy.

I have recently started a new project, Project D.O.R.F., and many of the other developers are new to Git. Thus, they have been making some mistakes and not utilizing Git to its fullest. Thus, I have created this guide using Project D.O.R.F. as an example. As you become more experienced with Git, you can figure things out on their own. Until then, this flow will get you a very long way.

Assuming you have Git installed properly, the first thing you need to do is configure Git. It is done like so.

$ git config --global user.name "Your Username"
$ git config --global user.email "your@email.com"

These two commands will configure your username and e-mail address for every Git repository you work on with that user account on that machine. Every commit in a Git repository has an author, so this is necessary to see which people are responsible for what work. There is also a git blame command that can show who is responsible for each individual line of code in a repository. If these settings are not configured, those features will not work. Alternatively, you can change these settings on a per-repository basis, but I’ve never had a need for it.

The next step is to go to GitHub and create a fork. First you go to http://github.com and register for it, as you would any other web site. There is no need to pay any money unless you would like to have private repositories that nobody else can access. Next go to the GitHub page for the project you would like to work on, in this case it is http://github.com/Apreche/Project-DORF/. Then, click the fork button. This will create a copy of the project in your GitHub account. You can see that lackofcheese has a fork of the project at http://github.com/lackofcheese/Project-DORF/.

Now that you have your own personal fork of the project, you need to actually get the code onto your local machine to begin working on it. This is done with the git clone command. Remember, you want to clone your forked repository, not the original one. You also want to clone using the SSH method. Follow the directions on GitHub for configuring your SSH key.

$ git clone git@github.com:yourusername/Project-DORF.git

This command will create a folder named Project-DORF which contains a clone of the repository. If you would like to put the repository into a different folder, feel free to move or rename it. Alternatively, you can do something like this which will put the repository in a directory named foobar

$ git clone git@github.com:yourusername/Project-DORF.git foobar

So now it’s time to get to work. Go into the folder and take a look at the files, and learn the project. You’ve got to read code before you can write it.

$ cd Project-DORF

You’ve read the code, and you’re ready to make some changes, but wait! What branch are you on? Let’s check.

$ git branch
* master

The master branch, you don’t want to be working directly on the master branch. Do all of your work on separate branches. And if you are working on more than one task, put each task in a separate branch. For example, let’s say you are planning to improve performance of the renderer, but also fixing a bug in a path-finding algorithm. Don’t do those things on the same branch. Do them separately. That is the power and magic that Git has to offer you, so there’s no point if you don’t use it. We’re going to fix a bug on a separate branch, let’s go.

$ git checkout -b bugfix
$ git branch
* bugfix
master

Perfect, we have a new branch. Now, start coding! Are you done coding? Time to commit. Let’s pretend we edited game.py, and we made a new file foo.py.

$ git status
# On branch bugfix
# Changed but not updated:
# (use "git add ..." to update what will be committed)
# (use "git checkout -- ..." to discard changes in working directory)
#
# modified: game.py
#
# Untracked files:
# (use "git add ..." to include in what will be committed)
#
# foo.py
no changes added to commit (use "git add" and/or "git commit -a")

It seems like git recognizes that we have a new file, but it’s untracked. It also recognizes we have modified game.py, but it’s not updated yet. You can see that there are instructions right there on the screen as to how to commit our changes. We can checkout a file to reset it to its pre-edited state. This command will undo our changes to game.py.

$ git checkout -- game.py

But we want to commit our changes, which means adding them first. We can explicitly add the files with the git add command.

$ git add game.py
$ git add foo.py

Then we can commit the changes with the git commit command.

$ git commit

Let’s say you only want to commit some of the changes, but not others? That’s perfectly fine. Just add the files you want to commit, and don’t add the files you don’t want to commit. Protip: Use the special commands like:

$ git add -i
$ git add -p

to interactively select line-by-line which changes you want to add. If you add two bits of code to a file, and only want to commit one of them, then only commit one of them. Nothing is stopping you.

The thing is, most of the time you are working, you aren’t making new files. You are just editing existing ones, and you want to commit all of your changes. In these cases use this shortcut command.

$ git commit -a

This will automatically add and commit all changes to files that already exist. You will still need to explicitly use the git add command to add in any newly created files. You will also need to use the git rm command to remove files. If you move or rename a file, just be sure to git add the new one and git rm the old one in the same commit, and git will figure it out, like magic. git rm is just like git add, so you have to commit afterwards. You should also use commands such as git add -A and git add -u to easily add many files at once. See the official Git documentation for more on that.

You’ve got your code in a separate branch from master, and you’ve committed your changes. Now it’s time to share them with the world. But first, you’ve got some work to do. While you were working, other people were also on the job. There is probably new code out there, and you have to make sure that your code plays nicely with it. The first thing you have to do is to add the canonical repository for the project as an upstream source. You only have to do this command once per clone.

$ git remote add upstream git://github.com/Apreche/Project-DORF.git

You don’t have to call it upstream, you can call it whatever you like, but upstream is an appropriate name. Regardless, this command creates the hook you need to be able to get updates from the “lead” repository. How do you get those updates? Let me tell you.

There are three commands you have to be concerned with here: fetch, merge, and pull. Fetch will get the new patches from the remote repository. Merge will merge those patches into your current branch. Pull does both at the same time. Let’s do it.

Go back to the master branch.

$ git checkout master

Fetch and merge changes from the upstream master into your local master.

$ git pull upstream master

Ok, so now your master branch has all the latest goodies that everyone else has been working on. But are your changes compatible? Let’s find out.

Switch back to your bugfix branch.

$ git checkout bugfix

Rebase!

$ git rebase master

Rebase can be a scary thing, but it’s not scary if you are careful. In this case, what we are using it for is pretty simple. It takes your changes that you have made on your bugfix branch, and sets them aside. Then it takes all the new goodies from the master branch, and puts them on your bugfix branch. Then it takes your changes and puts them back on top of that. If there is a conflict, it will provide you with instructions on how to fix it. Git has three different merging algorithms, though, so that rarely happens. Now that you are all rebased, test your branch and make sure it works. If changes are necessary, go back a few steps. Add and commit the changes. Make sure upstream doesn’t have any new changes. If it does, rebase again, and so on. Eventually, you will have a bugfix branch worthy of sharing with the world. Let’s do it.

$ git checkout master
$ git merge bugfix
$ git push

You’ve now pushed your changes out to your GitHub page. Congratulations. Now the rest of the world can go to your page and see those changes. Other people working on the project might even take your changes and start fiddling with them themselves. Don’t worry about that, though. You are interested in getting your changes upstream into the canonical repository. So go back to the GitHub web site for your repository, and click the pull request button. Send a pull request out to your collaborators, in this case Apreche, and they will be notified that your changes are ready for merging. Wait patiently, and if your patches are good, they will be part of the upstream. More information on pull requests is available at http://github.com/guides/pull-requests

Now let’s get advanced for a second. Git really doesn’t believe in the idea of a canonical repository. It’s just a concept we use to make things easier for ourselves. Someone coming to GitHub for the first time is going to see 10 repositories with similar code, how do they know which one is the right one? This is why picking one to be the one is a good idea. However, since there really is no such thing, you can actually do some fancy stuff. You might want to use the git remote add command to add links to some of the other contributors on the project. For example, lackofcheese might do this to be able to fetch amelim’s changes that are not yet available at upstream.

$ git remote add amelim git://github.com/amelim/Project-DORF.git
$ git fetch amelim
$ git merge amelim/master


It also might be a good idea to merge different contributor’s patches into separate local branches which you then merge together. Never be shy about making a ton of local branches. You can delete the ones you are no longer using to clean up cruft, and the cost of making them is very minimal. Don’t hold back, branch like a mad man. Here’s an example of how I would merge changes from two other contributors with my code and then push it out on the master branch. When I was done with this, I would send out a pull request.

$ echo "These remote add commands only need to be done once ever per clone"
$ git remote add ameleim git://github.com/amelim/Project-DORF.git
$ git remote add lackofcheese git://github.com/lackofcheese/Project-DORF.git
$ git checkout -b mycode
$ git add somefiles
$ git commit
$ git checkout -b cheese
$ git fetch lackofcheese
$ git merge lackofcheese/master
$ git checkout -b melim
$ git fetch amelim
$ git merge amelim/master
$ git rebase cheese
$ echo "The melim local branch now has amelims changes merged with lackofcheese's"
$ git checkout mycode
$ git rebase melim
$ git checkout master
$ git merge mycode
$ git push
$ echo "mycode on top of amelim's on top of lackofcheese's pushed out to the world"
$ echo "let's delete the branches we are done with
$ git branch -D mycode
$ git branch -D cheese
$ git branch -D melim

When you execute the rebase commands, you may be prompted to fix conflicts. If so, just follow the directions on the screen carefully. They usually involve editing the files in question with your editor to fix the appropriate sections by hand, re-adding, and re-committing the changes. It doesn’t happen often, and it’s not nearly as hard as it is to fix with non-distributed version control systems. Just make sure to test your code after resolving he conflicts.

That pretty much covers my basic work flow. Just to reiterate. Make separate local branches. Do work on them. Pull (fetch and merge) the upstream changes to your master branch. Rebase your changes on top of master then merge them into master before pushing them out to the world. If necessary, pull the changes of other collaborators into separate branches, and rebase/merge their changes together. Then take the sum of those changes and treat them as you would your own. Rebase them on top of the upstream master, merge them in, and push them out.

I have followed this work flow for quite some time, and it has served me well. Feel free to ask any question. Happy coding.

It all comes down to one thing. Why must we be forced to choose between a good user experience and openness? How come we can’t have both? Every open platform has a god-awful user interface, relatively speaking. All Apple products, especially the portable devices, have a great user experience. I don’t personally like OSX, but I can’t deny that it gets a lot of things right that everyone else gets wrong. Font rendering is a good example.

People who don’t care about openness, or aren’t aware of the problem, are thrilled. They get everything they want. Hooray for them. People who do care about openness are enraged. Why are they mad about something they aren’t going to buy or use? This is easy to explain.

They are mad because they see an unfulfilled potential. Think of a screwdriver. A typical screwdriver can screw any screw that is of appropriate size. Imagine a copper screw and a steel screw that are otherwise identical. For some reason the screwdriver works on the steel screw but not the copper one for no apparent reason. I think most people would consider such a screwdriver to be broken.

A device like the iPad has all the hardware necessary to do a billion amazing things. However, it only does a few thousand things. Why? Because Steve Jobs says so. Who is he to tell me what I can and can’t do with this thing that I have bought? If you compare it to the screwdriver example, you can say that the iPad/iPhone/iPod are all broken. Broken technology enrages the geek, especially when they can’t fix it.

The nerd goes out to get a different screwdriver that is not broken. All these other screwdrivers are great. They work on all screws, as expected. However, they all have really uncomfortable handles. Every single screwdriver out there which is not broken is really uncomfortable. There is not a single exception.

Why can’t someone just take the good screwdriver and stick it on the comfortable handle? It’s so simple. There is no law of physics preventing it from happening. It’s 100% possible, but for whatever reason it just doesn’t exist. Every single screwdriver on the shelf is either broken or uncomfortable. The geeks can not be satisfied. Their rage multiplies exponentially.

This is the source of the rage against Apple. Apple seems to be the only company capable of making screwdrivers with comfortable handles, but their screwdrivers can’t do half of the things that other cheaper screwdrivers can do. Meanwhile, all those other cheaper screwdrivers give you blisters when you use them. I’m getting angry just thinking about this imaginary screwdriver scenario.

There are only two solutions I see. One is that Apple opens up. Apple is clearly capable of open sourcing things. They could just open their portable hardware and software, and that would be that. Obviously it will never happen, but I don’t understand why. If Apple did it, they would have immediate complete market dominance. If they had a comfortable and fully featured screwdriver, all competition would crumble instantly. There would be no reason to buy any competitor’s product.

The other solution is for the open source people to just shamelessly start ripping Apple off. If we can’t invent something better, which we clearly can not, then let’s reverse engineer and straight-up steal all of Apple’s stuff. Take the screwdriver handle down to the metal shop, copy it exactly, and attach it to a real screwdriver. Patent laws my ass. Why should we settle for anything less than achieving our full technological potential? Patents are supposed to help, not hinder, progress. If they are hindering, let’s just ignore them.

Google, I’m looking at you. You must know that Android’s UI sucks compared to iPhone. You have a jillion dollars. Just hire the best designers in the world, and get it done. What are you waiting for? You’re supposed to be a company of geniuses, why didn’t you just do this in the first place for Android 1.0? What can you possibly be thinking? What are all the non-Apple companies thinking?

I think the primary thing that confuses people about Git is that it assumes that if you are developing software on a team, that you actually talk to people. Whether it’s by IM, IRC, a web site, Twitter, Git doesn’t care. It just assumes that you actually have some form of regular communication with the other developers.

You see, the non-distributed version control systems do not make this assumption. If you are using subversion, your work flow is dictated by the limitations of the software. You do some code. When you commit, you also share. Everyone commits to the same place.

With this sort of system, you don’t need to talk to the other developers to know what to do. There is only one canonical current version of the code in one repository. If you do an svn update, you know everything you need to know. If you change the code, you just commit, and you don’t need to tell anybody. They will see your changes when they eventually do an svn update.

This system inevitably leads to the many problems that Git solves. The one I always like to point to is the situation where you need to make a quick bug fix while you are in the middle of working on a big new feature. You need to make a version of your code that has the fix, but not the new feature. If you committed the feature, it’s going to be a nightmare to get a version of the code without it. If you didn’t commit the feature, it is very hard to collaborate with others. Branching solves it, but branching in subversion is slow and difficult, and it’s shared. You might ruin everyone else in the process.

Git lets you do absolutely anything that is possible to do with your code. You can branch, tag, commit, share, patch, and unpatch your code any which way you can imagine, and some you can’t yet imagine. As long as you are aware of all the commands and features, you can never dig yourself into a hole. You will always be able to deliver the code with the combination of patches that you need.

The problem with this is that since you have so many capabilities, there is no dictated work flow. Should you have a central repository? Should you push back and forth between individual developers? Should we have any shared branches? Should we use tags? Should you use soft tags or hard tags? Should you merge? Should you rebase? All of these questions, and more, are the subject of much debate.

I think that all of these debates are silly. The correct answer is that there is no correct answer. All of the features of Git are just tools to help you get your code the way you need it to be. If rebasing saves the day, then rebasing is awesome. If sending patches by email works for you, then that’s also awesome. Whatever process you come up with for your team to work together on the same code is fantastic, if it works for you.

Even if you decided to follow a subversion style of development, you can absolutely just do it with Git instead. Git can work exactly the way SVN works, if you wish it to. The best part is that you can use it exactly like SVN, but because it is Git, you can escape the usual SVN traps when they come up. Therefore, I see absolutely no reason for a new project to use SVN when you can simply use Git and have developers agree to work in an SVN flow.

And that is the beauty of Git, is that it lets you come up with your own process. How you work is no longer dictated by the limitations of the software you are using. It is decided by the developers themselves. And even if developers like to work differently, as many do, they can each work in a way that is comfortable for them without interfering with anyone else. As long as the team communicates with each other, and agrees on how to share code, all is well.

Just to drive the point home, Git was made by Linus Torvalds to use for developing the Linux Kernel. How do you submit a patch to the kernel using Git? You put your patch up in your Git repository, and you email someone requesting them to pull the patch from you. Email!? With SVN you just commit, no need to talk to anybody. I don’t know about you, but do you think that forcing code changes on other developers without having to talk about it a good idea?

If you actually bother to talk to people then 500+ forks is not a problem. You ask someone and they tell you what fork to clone. Just ignore all those graphs, they mean nothing.

However, I must admit that the 500+ forks is indeed a symptom not of Git itself, but github’s design, and users not understanding Git. Github makes the fork button very prominent, but forking a project on Github is not really something that people should do that often. Github is not Git. The design of Github, intentionally or unintentionally, leads users into doing something they don’t actually want to do. It doesn’t really help people understand Git, and it doesn’t help people communicate.

What most people should be doing is just cloning repository to their local machines, and making changes there. If they want to submit the changes, then they should use non-Git communication tools to talk to the person they cloned from. They only need to fork on github if they want to publicly share changes that have not yet been, or will not be, merged into the actual project. Otherwise, they should not fork, just clone.

If you do have changes that are not accepted, then sharing them publicly on github in a fork is a very useful thing. Maybe there is someone out there who needs a copy of djata with a slight tweak, but someone else made the tweak already. The tweak may never be accepted to the canonical codebase, but it can still be made available in a form. That might save someone the time of making that same tweak themselves on their own, which is what they would likely be doing on a project managed by subversion.

Right now I see people with SVN mindsets who won’t switch to Git because they don’t truly understand DVCS. Likewise there are people with SVN mindsets who don’t understand DVCS who switch to Git anyway for whatever reason. The missteps of the latter group provide many of the stories that will keep the former group scared away.

All in all, I’m not too worried. When I was in college, less than a decade ago, the CS department taught us RCS and the software engineering department taught us CVS. In my senior years some people starting using SVN, which was bleeding edge at the time. Git was released in 2005. The world of version control moves surprisingly rapidly. I think Git’s dominance is pretty much guaranteed at this point. It will just take time for programmers to escape the SVN mindset and understand the, admittedly more complicated, DVCS world.

We Git users are here to help, so ask before you run away or jump to conclusions.

HTML5 is on the move. Pretty much every browser other than Internet Explorer is supporting it pretty well. However, the codec issue on the video tag is going to be a major headache for Firefox. YouTube, which is really the only video site that matters, has decided to go with H.264. Firefox can’t include H.264 because of patents, and can only support Ogg Theora. Chrome and Safari do not have this problem. IE will also not have this problem, if and when they get their act together.

This is due to the balance of open source that people like RMS do not understand. If you demand everything be purely open source, you will be missing out on something. The same goes if you demand everything be purely closed source. Some applications only exist closed, and some only exist open. You will have to use both to get an optimal user experience, especially on the desktop.

The overwhelming majority of users do not care about whether something is open or not. They only care that it is free as in beer, and that it has all the features they desire. Regardless of speed benefits, Chrome will beat Firefox because of its willingness to not be 100% pure open source, and include things like the patented H.264 video codec.

For those of you who do not know, Ubuntu has a very strict release cycle. Every six months they release a new version of Ubuntu. The most recent release is 9.10, released in October 2009. It includes version 2.6.31 of the Linux kernel. If there is a bug or security patch to that kernel, Ubuntu will push out an update to the users, such as the current version 2.6.31-17. The thing is, the current version of the Linux kernel is actually 2.6.32.5. Ubuntu 9.10 users will never get version 2.6.32. In order to get a newer version of the kernel, they will have to build it by hand, or wait for Ubuntu 10.4 in April.

What’s the big deal? The kernel version barely makes a difference in the user experience. Most users will probably not notice or care, even if they are stuck with a very old kernel version. Well, what if you apply the same update policy to something like Firefox? Firefox 3.6 was just released, but Ubuntu users will not get that update. Windows and Mac users will, because Firefox updates itself on those platforms. Ubuntu users will have to wait for Ubuntu 10.4, or jump through hoops to update Firefox by hand. They have to wait 3 months for a very significant user experience update.

The problem with these two open source projects, and others, is not that they are open source. Open source is awesome, and it’s the only reason these projects can even compete in the first place. The problem is that unlike say, the Apache web server, these target every day users. However, their philosophies and policies do not make the experience of the user their number one priority.

Yes, there are good reasons behind these policies. I support stable software releases and being as open as possible. It’s simply that the user experience is even more important than either of those. You need to make exceptions to policy when the user is suffering. If a store didn’t make a reasonable exception to its return policy, you would give it a bad review and maybe even stop shopping there. Why can’t open source projects also make reasonable exceptions? If they don’t, they’ll just slowly lose ground to their competitors.

I remember when I was a kid, and landline phones were still huge. Sprint made a big deal about their ten cents a minute.

The number of cents you paid per minute was a huge deal. I remember my grandmother would dial a five digit number before making a long distance call in order to get the price down to five cents per minute. With truphone you can now make many international calls for under three cents per minute. Taking inflation into account, voice calls have gotten ultra cheap over the years.

Now, I’m someone who doesn’t talk on the phone much at all. I still need a phone number because it’s sort of necessary for engaging with society. I also do call family and relatives, and occasionally friends when necessary. But most of my calls are very very short.

So what am I paying for this voice service I hardly use? I have AT&T’s “Nation 450 Rollover & 5000 Night/Weekend & Unlimited Mobile-To-Mobile Minutes” plan. It costs me $39.99 a month. There is no smaller plan available. If you do the math, this comes out to under ten cents per minute. If I’m paying for just the 450 minutes alone, each minute is under 9 cents. Good deal right?

It would be a good deal if I talked on the phone. As it stands, 4135 rollover minutes available. Oh they’re so generous letting me rollover the minutes, right? Wrong. The rollover is just an excuse. It makes you feel less bad about paying for all these minutes you never use. The thing is, if you never use them, you’re paying lots of money for absolutely nothing.

On my last bill I used 20 of the 450 minutes, 4 unlimited M2M minutes, and 64 night & weekend minutes. Even if I were paying a ludicrously high price of 25 cents per minute, for all the minutes, my bill would have been $22. Instead, I’m paying 45 cents per minute.

The way they structure the plans is highly deceptive. You don’t think about your voice costs in terms of cents per minute anymore, because you are paying a fixed rate. But when you really think about it, you’re either paying for something you don’t use, or you’re paying way too much for the little that you do use.

Now, for someone who talks on the phone a lot, this probably isn’t a problem. If I actually used all those minutes, I would actually be getting a pretty good deal. It wouldn’t be an amazing deal, but certainly not a rip-off.

The other carriers are no better. T-Mobile and Verizon have similar minimum plans of 450-500 minutes for $40. Sprint has a voice plan of 200 minutes for $30. I think somewhere in the fine print there’s the option to not pre-pay for any minutes, and pay only for the minutes you actually use. Of course, they charge 45 cents for those minutes, making it moot.

There is an argument to be made that wireless minutes should be more expensive than land line minutes. That’s acceptable, but it’s not true. Remember, if I used all my minutes, I’d be paying a reasonable price for them. The problem is that the minimum tier is much too high.

I’m willing to bet if you looked at the books for any of these wireless companies you would see millions upon millions of dollars paid for unused minutes. I’m paying as much money for unused minutes as I am for unlimited data, which I use a lot. It’s about 30% of my phone bill, and I wouldn’t be surprised if it was 20-30% of their revenues.

Yes, text messages are a rip-off. Yes, data plans are secretly and evilly limited. But most of the money people like me are losing is paying for minutes we never use. Over the course of my two year iPhone contract, I’ve probably paid over $700 for unused voice minutes.

It’s nice that we’re actually seeing a little price war for the unlimited plans. These guys are actually starting to cut some of their ludicrously high margins on the high-end in order to push high-end smart phones to more customers. That’s well and good, but let’s also see some competition for the lower end plans. I want to see unlimited data and text for $30-40 with 100 voice minutes for $10-20. Even those prices are a little high, but they are far more reasonable than what I’m forced to pay now.

Maybe soon I’ll be able to ditch voice service altogether and use a VOIP solution. Perhaps the reason they don’t improve their data networks and expand coverage more quickly is to prevent me from doing that very thing.

If you accept that, then you must also accept that open source is the only environmentally responsible method of software and hardware development.

A month ago I went to the Boxee Beta unveiling in Brooklyn. I asked Avner Ronen why the Boxee Box didn’t include Cable Card, DVR, Blu-Ray, etc. Because the Boxee Box only ran Boxee, it would increase the number of boxes under a TV, not decrease them. He agreed that there was definitely a box fatigue, and he also wanted fewer boxes under his TV. He expressed hope that another electronics manufacturer would produce a machine that would run Boxee and all of those other applications simultaneously, but the Boxee Box would not be that device.

There was a time where typewriters were obsolete, but PCs hadn’t yet become affordable, there was a product known as a word processor. All that machine did was process words. Imagine if today you bought one computer that only ran Microsoft Word. Imagine if you had to buy a separate machine for iTunes, and a separate machine for web browsing, and so on. It’s so obviously ridiculous, even normal people would laugh. You have a general purpose computer that is capable of running a wide variety of software. There is no reason to have specialized hardware for a single piece of software for the vast majority of applications, especially consumer applications.

As you are reading this right now, I want you to think about the boxes you have under your TV. I imagine that most people have a cable box, some game consoles, a DVD or blu-ray player, a DVR, maybe a Slingbox, and maybe even an old VCR. Some of these have been combined. Many cable boxes have a DVR in them. Game consoles tend to include DVD or Blu-Ray players. That being said, people I know have a lot of boxes in their primary entertainment centers.

This is just as ludicrous as having a separate computer just for word processing. All of those boxes are general purpose computers. There is absolutely no reason that you can’t have one box that is a cable box, DVR, DVD/Blu-Ray player, game console, and all around media handling machine. How come such a box doesn’t exist? Sure, some nerds have built such boxes on their own, but there aren’t any that are available to consumers on the mass market.

The reason is closedness. Cable boxes are closed. DVD and Blu-Ray decoding is closed. Game consoles are closed platforms. A DVR doesn’t have to be closed source, but many things it deals with, like video codecs, DRM, and other media protocols, are closed. Even VCRs have Macrovision DRM to deal with.

To protect the closed nature of all of this software, it is delivered to the user in a black box. They provide unopenable and unmodifiable hardware with the software built-in. If you want to add any features, you need to get more boxes or different boxes. Sometimes those other boxes do not work well in concert with the black boxes, and you get duplicate functionality. This is ludicrous and technologically unecessary.

The only benefit of this system is that it gives the media companies some slight protection against piracy and such. As I’m sure you are well aware, it is working really really well. No matter how hard I try, I can’t find any high definition movies or TV shows available for free. Man, it’s so worth wasting all this extra energy on electronics to prevent piracy.

The benefits of opening all these systems are that consumers will have an infinitely better experience, and we will reduce a tremendous amount of waste. There is no reason that we all can not have one single computer under our TVs that runs any and all living room entertainment software.

Game console makers should even pack it in. Their insistence on making games for proprietary consoles is silly. There is no reason people can’t just make PC games that have different modes when hooked up to a TV instead of a monitor. Gaming controllers like Wiimotes (unofficially via Bluetooth) and XBox 360 controllers (officially via USB) already work with PCs just fine.

If all of this software were open source, then everyone would be able to have one, and only one, box under their televisions. That box would handle everything from watching YouTube to watching the pre-recorded evening news. The software on it could be upgraded almost perpetually to handle any new applications, like 3D movies. You might not even need a power strip behind your TV. Heck, you could reduce the audio and video cable rats nest to just one or two wires. We would all save a lot of money and space in the landfill thanks to reducing our box count. Not to mention our entertainment centers would be a lot less cluttered and more aesthetically pleasing.

Also, be aware that I’m just using the home entertainment system as an easy example. There are plenty of other areas in which lack of openness increases our need for extra devices. Almost every electronic device you see nowadays has a general purpose processor inside. If it does, then it is capable of handling far more functionality than the built-in software allows. Even for all of the things an iPhone can do, the piles of rejected apps are clear evidence that the hardware is capable of far greater functionality than the software allows for. You can also see clear evidence by looking at the wide range of devices which have been made to run Doom and/or the Linux kernel.

If all of these things were open source, we could exploit every piece of hardware to its fullest potential by modifying the software. A black box is almost always going to fall short of achieving its full potential. It will always have possible capabilities that go unused due only to lack of software. If that capability is desired, it will only be achieved through the production of yet another piece of hardware, and expenditure of more money and energy.

Waste not with open source.

And the problem is pretty obvious. Companies want to make tablet PCs, but they don’t know the use cases. So far, tablets have just been crappy laptops without keyboards. That’s because the use cases that the manufacturers have in mind are the exact same use case that a laptop has. The result is devices that have the same use case as laptops, but are crappier than actual laptops. This is why tablets have been utter failures. The best they could come up with was that maybe a doctor would want one because it could be used while standing up near a patient’s bed.

Also, the original tablet computers all had resistive touch screens. The capacitive touch screen didn’t really exist yet. Everything was designed around a stylus without multi-touch. You couldn’t type with two thumbs, pinch, scroll, etc. They also had no accelerometers or other sensors. Designing around those technologies prevented them from making a worthwhile device, even if they had the right uses in mind.

Even today, people are only starting to get the right idea about the uses for a tablet PC. People are suggesting to use them as remote controls for home media centers. That’s a good start, the <a href=”http://www.pepper.com/”>Pepper Pad</a> was doing it years ago. Some are using them as e-readers. You could even argue that the Kindle and other e-readers are actually tablets. Even so, these limited uses are not enough to justify the expense of a tablet PC when you can just get a Logitech remote control and a netbook.

It’s looking like the long awaited Apple tablet is finally moving from rumor to reality. I anticipate that this iSlate, or whatever it will be called, is going to satisfy one of the two worthwhile use cases for tablet computing, but there is still one other use case that has been completely ignored. That second use case offers a land of opportunity for any electronics company that is willing to go for it.

But first, let’s discuss Apple’s tablet. Where could it fit into the lives of consumers? Well, I don’t know about other people, but I can tell you where it fits into my life. Every morning I am awoken by the alarm in my phone. I pick up said phone and check it before getting out of bed. I then go through my morning routine on my desktop. Then I go to work, using my phone in transit if necessary. I use a desktop/laptop at work. I return home to use my desktop for personal work, like this blog post. If I go out to a friend’s house or such, I bring my laptop for when my phone won’t suffice.

Where does the tablet fit in? It fits in for comfortable non-work home computing. When I wake up in the morning and check my phone, that kind of sucks. Even if you have an iPhone, it is small and harder to use than something with a bigger screen. That lack of comfort and size are exchanged for portability. But if I’m in bed, I don’t need portability. My laptop can get the job done, but it sucks to use a laptop in bed. I have to sit up and type. I only need a keyboard if I need to work, not to read some emails and web comics.

If I’m eating lunch, and I want to read some news while I eat, I have to sit at a table or desk, upright, in front of a desktop or laptop. There is no way I can lay down and recline on the couch while simultaneously browsing the web comfortably. Sure, I can use my phone, but again, it is tiny and uncomfortable.

This is the use case that Apple’s tablet is going to fulfill. You’ll use it in bed, on the couch, or anywhere else around the house you just want to be comfortable and read the web. I’m probably not going to get one, due to cost and closed Apple-ness, but I will probably get a competing device when they inevitably appear within a year.

So what’s the other use case for a tablet PC? If you have been paying attention, ever since tablet PCs hit the scene, only one group of people have wanted them very badly. Go and search the web for tablet PC discussion groups. Who will you find in there? Artists, that’s who.

Tablet PCs are perfect for artists. Most artists get by using Wacom tablets to draw in the digital world. The problem with them is that you look at the screen and draw on the tablet. It’s much more natural to draw directly on the screen, simulating drawing on real paper. Wacom does make a product called the Cintiq that does just that. The problem with the Cintiq is that it is incredibly expensive, and it’s basically a monitor. It is not portable at all.

What someone needs to do is make a tablet PC with a Wacom-powered screen. Make it just powerful enough to run Photoshop nicely, comfortable to hold, and at a good price. Artists will buy it like crazy. Right now these artists have to huddle over their desks, or carry around lots of equipment in their bags. Making a Wacom tablet computer will free them, and they will thank their saviors with piles of money.

Anyone out there who is looking at the tablet world, and trying to make sense of it, just think of the two use cases. Tablets will allow people to compute comfortably from their bed, couch, back porch, etc. They can also help out digital artists a great deal. If you find yourself wanting a tablet, think about what you are actually going to use it for. You’ll easily see how pointless existing tablets are, but you’ll also realize the potential of what they could be.

The fundamental problem here is that people want fancy phones. The thing is, fancy phones cost a whole lot of money. If you pay full retail price for an unlocked smartphone, it can cost up to $1000, depending on the phone. You can get a really big TV for $1000. Most people, in the US at least, won’t pay that kind of money for a tiny phone, even though it is something they use constantly.

The solution is to subsidize the phones. Sell the phone for $200 or $100. Then, simply require the customer to stick with your service for two years. Over the course of those two years, you will make back the cost of the phone. If they leave early, you’ll take a loss, so charge them an early termination fee. Makes perfect sense, on paper.

In reality, it just doesn’t add up. Let’s say I already own an unlocked phone. I go to a carrier and ask for service. Even if I don’t get a contract, I’ll be paying the same data and voice rates as someone who had their phone subsidized by the carrier. Shouldn’t my monthly fees for data and voice be less? The carrier didn’t subsidize a phone for me, but I still have to pay them just as much money as if they had? If I stick with them for two years, then I’ve paid for a phone subsidy without receiving a heavily discounted phone! People who did not have their phones subsidized should pay less monthly than those who have.

Also, keep in mind that early termination fees never reduce to zero. The fee exists, supposedly, only because of the phone subsidy. Therefore, the cost of the ETF should be equal to the remaining unpaid value of the phone. Let’s say I get a phone and $400 of the price was subsidized down to $0 with a two year contract. 400 divided by 24 is about $17. The fee in the first month of service should be $400. The fee should then decrease by $17 each month, and in the final month the fee should be $17.

Wait, this payment structure sounds familiar. That’s right, it’s called a loan! Actually, it’s a 0% interest loan. It’s also exactly what mobile phone carriers need to replace subsidies with.

Step one is to lower all voice/data/text charges. They are absurdly high. Because there will be no more subsidies, there will be no more need for that kind of ripoff. Existing users with subsidized phones will still pay the old fees, or early termination fees, to grandfather them into the new pricing structure.

Step two is to advertise the true cost of phones. Let everyone know the full MSRP of each and every phone. This allows the customer to make a much more informed decision when they select a phone. It also gives people the option of buying phones without service. Carriers won’t like this, because most people will opt for the cheap phones once they realize the trust cost of the expensive ones. Those cheap phones don’t require the data plans and other features that the carriers also like to charge an arm and a leg for.

The final step is to offer financing. Even affluent people don’t want to shell out that much money on a phone all at once. Instead, the carrier offers financing at zero or a very low interest rate, obviously depending on the customer’s credit rating and such. The monthly payments for this loan will be included in the monthly bill for voice and data. Even if someone cancels their voice and data contract, they will still have to pay off the device itself, or return it.

This is really a win win solution. Carriers will save money from subsidies, and maybe make money with a small and reasonable amount of interest on the financing. Customers will have the freedom to switch carriers as they please, and pay reasonable prices for voice and data service.

The last piece of the puzzle are the device manufacturers. Because of carrier lock-in and subsidies, most handset manufacturers are really selling their product to carriers, not to customers. Motorola and LG make a handset that Verizon loves, not one that you and I love. I think that Apple has demonstrated with the iPhone that by creating a handset for the customer, you can make the real bank. If subsidies go away, perhaps we will see some real competition between the manufacturers for the consumer’s love. That should result in the current phone arms race escalating and resulting in better cheaper devices for everybody.

Kill subsidies. Get rid of the contracts. Charge full price for devices, but allow financing. Problem solved. Do it now.

The first decision I made was to build the site with Django. It has been my web framework of choice for many months now. I used it to build the new frontrowcrew.com, and I’m very satisfied. The thing is, Django wasn’t always usable.

Some months ago Django 1.0 was not released yet. Yet, when you went to the Django web site and community, it seemed as if nobody was using 0.96, the stable version. When you go to the Django documentation online, it defaults to showing you the documentation for the version currently in development, rather than the latest stable version.

If at that point in time you wanted to start a new project with Django, you had a tough decision to make. Do you start with 0.96 and stick with it? Do you start with 0.96 and go through the hassle of porting over to 1.0 later? Do you deal with the hassle of building on the development version and build your way up to the actual release? Do you just not start your project and wait for the release? This is a difficult decision to make, and there’s an easy answer I left out. Maybe you just shouldn’t use Django, and go find something else that isn’t in a state of flux?

Happily, Django 1.0 has been released now, and this is no longer a problem. Well, at least it isn’t a problem for Django itself, but it is a problem for many things built around Django. Take Pinax for example. Pinax is a project that builds on top of Django. They implemented a lot of solutions to things that are common to all web sites, such as registration, password management, blogging, friend, and tagging. My new site idea will need some of those things, and I want to reinvent as few wheels as possible, so Pinax was very appealing to me.

The problem is that Pinax is currently in the state of flux that Django used to be in. There are sites out there which are built using Pinax, so it can be done. However, the current stable version seems to have been largely abandoned by the community. In my limited experience, the development version appears to be far superior, but it’s still in development. Worse, as a newcomer to the Pinax project, their documentation is very unreliable. I find myself asking those same questions all over again. Maybe it’s easier to reinvent those wheels than it is to deal with the hassle of closely tracking someone else’s project?

Let’s say I just want to add tagging to my Django application. Why should I bother with Pinax, I can juse use django-tagging. It has a stable version 0.2 that came out, over a year ago? The site says that version 0.3 is targeting Django 1.0. In other words, it’s completely useless. I’m not going to use Django 0.96 just so that I can use the stable version of django-tagging. I’m also not going to go through the hassle of using an unstable version of django-tagging 0.3 from their SVN trunk. Since their project was dependent on Django, it was their job to follow Django’s SVN trunk and have django-tagging compatible with Django 1.0 the day that Django 1.0 was released, or very soon after. Their failure to do so has made their project useless.

Just for kicks, let me talk about Laconica. Laconica is an open micro-blogging platform. Think of it as a Twitter clone, but with a different, and better, system architecture. I’ve been running an installation of it for our podcast community, and it’s very nice for the users.

The problem with Laconica is that it’s a nightmare on the back-end. The project has noble goals. They want it to be as easy to install, upgrade, and administer as WordPress. And while they have the first verison of a web installer, the underlying architecture is frighteningly bad. You have to run a set of daemons to handle the queueing of in and outbound messages. That kind of design is going in the exact opposite direction of making things easy. What’s worse is that in the latest version they added even more daemons, and two of them decided to eat up all my CPU. Because it’s under constant development, there are frequent updates with awesome and significant improvements. That’s very good, but the upgrading process is nightmarishly difficult. I want to keep the Laconica server running for the users who enjoy it. It’s just that the administration is far more work than it’s worth.

Here’s the one piece of advice I can give to open source project managers out there. Make sure to maintain a degree of separation between your development community and your user community. A very small percentage of the people who want to use your project actually want to get involved in working on the project itself.

My web site is going to use Linux, Ubuntu, Apache, Python, Django, numerous Python modules, CouchDB, Postgres, memcached, and many other open source projects. If I’m working to build the site, I need all those things to “just work.” I don’t have time to get heavily involved in the communities for any of them. Any effort I have to spend tracking development versions, or dealing with bugs in someone elses code, brings me closer to the point where I would have been better of building it myself.

Actually, let me offer one more piece of advice. Lots of small easy upgrades are better than a few big ones. The difference between Pinax’s stable and development versions is enormous. What if instead of having that huge gap, they had instead made many small upgrades to the stable version over time? The gap between stable and development would be much smaller. Using the stable version wouldn’t be such a bad idea after all, and upgrading to newer stable versions wouldn’t be as big  a deal.

I’m a programmer trying to do a project on my own. I’m smart and full of enthusiasm for the latest and greatest open source goodness. I want to do everything the right way. I want to utilize as much open source code as possible, so that I only have to write the code specific to my project. Sadly, I am unable to do that if the open source projects are in a state of flux, or can’t be relied upon. When reusing code requires just as much effort as rewriting code, you know something is wrong.

First, I copied my favicon.ico to my S3 bucket. It’s just a 16×16 png file that I renamed.

In my settings file I made sure the MEDIA_URL was set to my S3 URL.

Then I made sure these things were imported in my urls.py

from django.conf import settings
from django.views.generic.simple import redirect_to

Lastly, I added this rule to my urls.py

(r'^favicon.ico$', redirect_to, {'url':settings.MEDIA_URL+'favicon.ico'}),

That’s it!

Let me break this down by generation. The first generation, 1g, was analog cellphones. You know, those old gigantic things that had a battery that lasted 5 minutes. There were many different standards for analog cellphones, but they all worked on the same basic principle. They encoded your voice into an analog wave, and transmitted it using a very high powered radio to the cell tower that would receive it and connect you to the phone network. The reason the battery lasted for such a short time was because the radio used so much power. The analog network was really only capable of transmitting voice, just like the regular telephone network, but you could theoretically use dial-up technology to transmit data over it very slowly and unreliably.

The main difference between 1g and 2g was that it moved from being analog to digital. The radios used a lot less power, audio quality increased, and you could send data as well as voice over the network. 2g is just better than 1g in every way.

The thing about 2g is that there are many different digital standards that work differently and are not compatible. Some phone carriers chose to use one set of digital technologies, and others chose a different set. Most of the world chose to use GSM, GPRS, and EDGE as 2g standards. Companies like Verizon, Sprint, and a few others chose to use CDMA, and 1x. That is why AT&T phones work in Europe, but Verizon phones do not.

Another thing is that the 2g phones that use GSM, GPRS, and EDGE have SIM cards, while the CDMA and 1x phones typically do not. A SIM card is a little plastic card that holds all your account information. In Europe, where most phones are unlocked, you can take your SIM card and put it in any phone. That phone is now your phone. If someone calls your number that phone will ring. This gives you the ability to change phones at any time. You can also do cool stuff like let your friend borrow your phone to make a call on his own dime when his battery ran out.

In the US, many people use Verizon and Sprint, who do not have SIM cards in their phones. Even people who use AT&T or T-Mobile, who do use SIM cards, have phones that are locked. If a phone is locked, it will only work with one particular SIM card. So even though you can take your card out and put it in an unlocked phone to make a call, you can’t have someone else put their card in your phone. This is why people fight so strongly to get their phones unlocked. The one downside of having phones unlocked is that it suddenly becomes a really good idea to steal expensive phones. I think that is a small price to pay for the convenience, freedom, and power for the consumer.

So 3g is all the rage these days. 3g simply refers to the third generation of wireless technologies. That includes things like HSDPA and EV-DO. These are still digital wireless technologies, like 2g, but they are capable of very fast data transfers. They aren’t as fast as a wifi, but they are so fast you can watch youtube over them if the signal is strong. There are many things you can buy that allow laptops to connect to 3g networks, so they can get really fast data service if no wifi is available.

Verizon and Sprint use EV-DO, while AT&T and T-Mobile use HSDPA. When people talk about the iPhone using 3g, they are talking about HSDPA. Apple just says 3g, because it is a better term for marketing purposes than a complicated thing like HSDPA.

The one last thing to talk about is 4g. 4g is the future. 4g basically kicks all ass, on paper. Theoretically we will be able to get super fast data everywhere. People are talking about speeds of 100Mbits minimum. We will see in the future what the reality is, but there is one thing very interesting about 4g. It seems as if Verizon, AT&T, and T-Mobile have all chosen LTE as their 4g technology of choice. When the 4g comes, we may finally see all the major carriers in the US using the same technology and delivering compatible phones. With Verizon promising any phone, any app, 4g finally might fix our troubles.

For any of you people out there who are getting ready to comment, yes I know that technically UTMS( HSDPA) is 3.5g, and that EDGE is 2.75g, but the purpose of this was to help people attain a colloquial understanding of the terminology, not to be super precise and confuse them further.

I hope I succeeded in helping people understand the weird cellphone acronyms. I just see so much confusion online, I had to do something.

There is another side effect of this that isn’t immediately obvious. Existing open source applications that are licensed under the GPL, and other similar licenses, can not be legally ported to the iPhone. If you make modifications to a program under the GPL, you must open source your derivative code. The Apple agreement prevents you from doing this. Therefore, you can’t port emacs to the iPhone without violating either the GPL or the agreement with Apple.

In the wake of the recent court ruling upholding the Artistic License, we are more sure than ever that open source licenses are enforceable. Thus, the only chance we have of seeing many open source apps ported to the iPhone is if Apple makes a change, because the GPL isn’t going anywhere.

Luckily many open source projects are licensed under the BSD, MIT, or other similar licenses. Projects under these licenses can be ported to the iPhone because the derivative code does not need to be open sourced. This is why we are seeing iPhone applications that contain code from PuTTY, OpenSSH, OpenSSL, etc.