depthq - ccm|mac

In light of the recent earthquake in Sichuan, our website team turned most of exoweb.net gray.

I just want to spend a bit of time to document how we did it.

First we change all the images into grayscale using PIL with the following code snippet:

import Image
import glob
import os

for file in glob.glob('*'):
    if os.path.isfile(file):
        print file
        img = Image.open(file)
        gray = img.convert('L')
        gray.save('gray-' + file)

We don't have a very complicated directory heirarchy, so the above code was sufficient.

We cloned the main CSS file and created a copy for the gray version. We remove the styles that used colours and replaced them with gray versions.

After that we updated our Apache rewrite rules and redirected the normal CSS and images to the gray ones.

As a parting note, I send my condolences to those who lost their loved ones.

I am finally fed up with the bright blue colour in Terminal and decided to change it. I guess I was able to tolerate this stupidity for quite a while. I remember Wang Chun tried this before.

But his tip is not complete for Leopard.

I found a tip here that made it work.

In summary:

1. Download and install SIMBL. I used this version and didn't need to configure anything.
2. Download and extract TerminalColours in to the right folder ~/Library/Application Support/SIMBL/Plugins
3. You might still get an error message, which is invoked by the previous installation, I deleted it from /Library/Application Support/SIMBL/Plugins/TerminalColors.bundle
4. Open Terminal, under Settings -> Text, choose "More..." and remove that irritating blue and select a colour of your choice
5. There might be other problems, check the comments section of that blog entry to learn more of what others went through

I downloaded a Korean movie, Changing Partners (official website) and to my surprise it was a .rmvb file. I personally prefer to play movies with VLC but this .rmvb file format is not supported. I need to wait for the next version of VLC but I want to watch this movie as soon as possible, possibly now or very soon.

I am kind of disgusted by software from RealMedia partly based on personal experience and partly based on what I read online.

I took some time to search for alternatives to run on the Mac OSX and I didn't find any. I know if I am running M$ Windows, I can use Real Alternative but no luck for me here.

Okay, fine, I'll download and install software from Real Media because I am desperate for some entertainment. I downloaded the Real Player 11 Beta and it was a whopping 13.3MB download. Alright, for broadband users this kind of file size is nothing but I am always amazed at software that is small and have tons of features. I guess the stereotypical example from me would be uTorrent. Why did I just mention uTorrent? I guess I just missed it since I am not using M$ Windows much these days.

Anyway, I got Real Player installed and tried to play it and after a few minutes I realised I didn't understand a word. It was all gibberish to me. They are talking in Korean! What a surprise! Okay then I need to look for subtitles. So I went to scour the Internet and all I found was websites waiting for me to download their malware so they can infect my Windows machine that I didn't have.

Hmm, what do to now? I looked into the folder that contained the downloaded files for the movie and there was a .rar file, waiting for me to uncompress it. Darn, there is no native support for rar compressed files in Mac OSX.

Oh yes, I remember that I have MacPorts installed, maybe I can find and install a command line utility for this. I did a search using the port search rar command and then found unrar utility and installed it using the sudo port install unrar.

I uncompressed the .rar file. Lo and behold there was a .srt file inside. I am bursting with joy until I realised that I have no idea how to load a subtitle file in the Real Player. I found a resource at captionkit that says how to do it but it didn't work (in fact that tip was for something else). After a while with more research I realised that the .rmvb file requires another 2 more files (.rt and .smil) for subtitling to work.

I googled for .srt to .smil converters but didn't find any. After a while I stumbled upon a few posts in the Doom9 forums that gave me hints on how to write my own .smil and .rt files.

So that is what I did, with some documentation provided by Real Media, I wrote some .rt and .smil files and tried to play it with the Real Player. It worked pretty well. If you know HTML, .rt and .smil files would look really familiar. Once I am pretty confident of my RealText knowledge I wrote a .srt to .rt converter in Python. The code is not very pretty and elegant but it works (feel free to use it and hack on it):

import sys 

f = open(sys.argv[1])

print '<window duration="02:00:00" width="640" '
      'bgcolor="black"><font color="white"><center>'

while True:
    subtitle_number = f.readline()
    if not subtitle_number:
        print '</center></font></window>'
        sys.exit(0)

    time_data = f.readline()
    begin, _ = time_data.split(' --> ')
    begin = begin.strip().replace(',', '.')

    print '<time begin="%s"/>' % begin

    subtitles = []
    while True:
        subtitle = f.readline().strip()
        if not subtitle:
            break
        subtitles.append(subtitle)
    print '<br/>'.join(subtitles)
    print '<clear/>'

The .rt file looks like this:

<window duration="00:56:00" width="640" bgcolor="black">
  <font color="white">
    <center>
      <time begin="00:00:40.273" end="00:00:43.008"/>
      Presented by Cinema Service Co., Ltd.<br/>
      In association with CJ Entertainment Inc.
      <clear/>
      <time begin="00:00:43.009" end="00:00:44.669"/>
      Produced by Cine2000
      <clear/>
      <time begin="00:00:46.046" end="00:00:49.879"/>
      Executive Producer<br/>KANG Woo-suk
      <clear/>
      ...

And this it not the end because a .smil file is needed to package and provide layout for the text stream (subtitling) and the video stream (the movie I want to watch so dearly). And the .smil file that I used looks like this:

<smil xmlns="http://www.w3.org/2001/SMIL20/Language" 
      xmlns:rn="http://features.real.com/2001/SMIL20/Extensions">
  <head>
    <layout>
    <root-layout backgroundColor="black" width="640" height="330"/>
    <region id="video_region" z-index="1"/>
    <region id="text_region" height="50" bottom="0" z-index="2"/>
    </layout>
  </head>
  <body>
    <par>
      <video src="postx-cp-a.rmvb" region="video_region"/>
      <textstream src="postx-cp-a.rt" region="text_region"/>
    </par>
  </body>
</smil>

Once you have the .smil, .rt and .rmvb (obviously) files you are ready. Fire up your Real Media Player and then open the .smil file and you'll be able to watch your movie with subtitles.

There are still some problems though. I didn't spend the time to figure out transparent background for the text region for subtitles. And the subtitles will stay on the screen until the next one pops up. So some subtitles will stay on the screen for a long time if no one says anything. In fact there is a way around this but adding the feature to make it disappear at the correct time (.srt files have this information) will create another bug: some subtitles won't even show up. So this is actually a feature of my .srt to .rt converter so that all subtitles appear.

You also need to do some manual .smil and .rt hacking for different movie sizes and duration. If you don't have the correct values, it won't play nice.

Phew! That was probably one of my longer posts. I am thirsty. Hope someone enjoys my little adventure in subtitle hacking. I guess its not something you do every time you watch a movie.

Hmm, I still want to write more actually. I just want to say that I also used my .srt to .rt converter for another movie, this time a Chinese one and it worked fine! God bless programmers!

Lets say you are in the midst of developing this new web 3.0 website that is going to have all the most sought after features of Gmail, YouTube, Flickr and Digg. You have a team of developers are trying to crank out as much code as possible.

Suddenly someone checks in some code and breaks everything. Nothing works and nobody knows why. You have to spend the time to identify the problem and fix it and things go back to normal. And shortly someone does that again and all work stops to a halt. Nothing works. Again, effort spent to find the problem and fix it again. Some angry screams were heard and a lot of lost time and effort.

All you want to do is to write some code and deliver a functionality. But so many things seem to get in your way. And by the way, the coffee in the office sucks. It almost doesn't seem worth it to write software. But there is definitely a better way to do this.

I've kinda been in this situation before and its really nerve wrecking when testing a software that is developed without unit testing. I was naive and dumb once (okay maybe twice ). You basically have to remember all the different things you can do with your software and test it a million times and tell yourself its all okay before you can relax and calm down. You lose a bit of your life every time you do it.

Its almost the stuff that is being written about in dailywtf but its happening to YOU.

Developing software in Exoweb is like the total opposite of the dailywtf. I would call it the programmer's heaven. I am exaggerating but we're pretty close. Once you start working here you'll be introduced to unit testing, coding conventions, code reviews, pair programming, scrums, dynamic languages, open source and so on and so forth.

Just before I even thought of wanting to jump on the Exoweb bandwagon, I only started reading more about software development, mainly through Joel on Software. I was hooked on reading all his articles and found more and more resources online that really expanded my knowledge. I did learn a lot through reading but you can only know so much and you'll have to start doing.

Working in Exoweb gave me a good place to learn up all these good software development practices. Its almost 2 years in Exoweb now (about 3 more months to go) and I think I've only touched the surface. There is so much more.

A thought just struck me today about how we're developing software in Exoweb: Its almost like its sugar-coated. Its all sweet and nice.

So why do I say this? As a programmer the most important thing is to write code to implement a functionality that will make the user happy. I'll give you an example. If I have to worry about making the test framework better, then I am not spending time fixing the customers problem.

In the current project that I am in, we don't really have to worry about that because a lot of effort has already been spent on this and we just need to use it. Just create a file with the correct suffix e.g. _stest.py, run one command and voila your test code that will test this super new cool feature in that new sexy AJAX form will just work.

Another pain point would be to update the production systems. Most developers would not need to worry about all the work and hassle that is required to deal with updating the production systems to deliver the functionality to the users. Most of the time the main worry is to ensure that the code does what it does and passes all the regression tests. I wish production update is as easy as running the svn up command but we're definitely far away from that goal.

We also have a very good culture of experimentation. There are a number of teams that are experimenting with agile methods and this means we focus on quick results, high productivity and light-weight methods & practices. Again, we can draw a lot of thoughts and experience from those that have done it.

I was a total noob on open source when I joined Exoweb and I still think I am but I've definitely learn more about software, open source or non open source, in Exoweb than I've done anywhere else in the same period. I've even managed give back a bit to the Django community.

I suppose this reminds me a lot about Joel's article about the developer abstraction layer. I really think that the piece is very insightful but its talking about slightly different things. I think Exoweb really deals with a lot of these small issues pretty well like making sure there's enough drinks, providing free snacks, having nice parties and events, listening to our feedback, thinking about the temperature and working conditions, ensure that networks is up and running, making sure that the new spanking machines run well with our development environment and dual monitors, hiring people that are smart and that is able to get things done, celebrate our birthdays and working anniversaries, making sure the coffee is good and popcorns are popping during the movie night and etc.

I think I've written enough of my random thoughts. What are your experiences and what have you learn from it? Do share and leave some comments.

NOTE: I tested this on SVN 1.4.2 and SVN 1.4.4

SVN will have problems when you rename a directory that is in a directory that you want to rename in the same commit.

I created the following directory hierarchy in SVN repository: dir1/dir2/dir3

I did the following:

Renamed a subdirectory

silvia:co1 cheeming$ svn mv dir1/dir2/dir3/ dir1/dir2/dir4
A dir1/dir2/dir4
D dir1/dir2/dir3/file3
D dir1/dir2/dir3


Renamed a parent directory, had to use --force

silvia:co1 cheeming$ svn mv dir1 dir5
svn: Use --force to override this restriction
svn: Move will not be attempted unless forced
svn: 'dir1/dir2/dir4' has local modifications
silvia:co1 cheeming$ svn mv dir1 dir5 --force
A dir5
D dir1/dir2/dir3/file3
D dir1/dir2/dir3
D dir1/dir2/file2
D dir1/dir2/dir4/file3
D dir1/dir2/dir4
D dir1/dir2
D dir1/file1
D dir1


Commit it, but get a weird error message

silvia:co1 cheeming$ svn commit --message 'test broken commit'
Deleting dir1
Adding dir5
Deleting dir5/dir2/dir3
Adding dir5/dir2/dir4
svn: Commit succeeded, but other errors follow:
svn: Error bumping revisions post-commit (details follow):
svn: Directory '/Users/cheeming/Work/TestSVN/co1/dir5/dir2/dir3' is missing


Check the status...hmmm

silvia:co1 cheeming$ svn st
A + dir5
A + dir5/dir2/dir4


Yes, SVN did warned me but I still don't know what is the extent of the error. There were no explanations on the consequences. So what am I supposed to do now?

I checked the log and it seems correct

silvia:co1 cheeming$ svn log -r HEAD --verbose
------------------------------------------------------------------------
r3 | cheeming | 2008-01-26 10:46:49 +0800 (Sat, 26 Jan 2008) | 1 line
Changed paths:
D /dir1
A /dir5 (from /dir1:2)
D /dir5/dir2/dir3
A /dir5/dir2/dir4 (from /dir1/dir2/dir3:2)

test broken commit
------------------------------------------------------------------------


I read somewhere on the web that I should do a cleanup

silvia:co1 cheeming$ svn cleanup
svn: In directory 'dir5/dir2'
svn: Error processing command 'committed' in 'dir5/dir2'
svn: Working copy 'dir5' locked
svn: run 'svn cleanup' to remove locks (type 'svn help cleanup' for details)
silvia:co1 cheeming$ cd dir5
silvia:dir5 cheeming$ svn cleanup
svn: In directory 'dir2'
svn: Error processing command 'committed' in 'dir2'
svn: Working copy '.' locked
svn: run 'svn cleanup' to remove locks (type 'svn help cleanup' for details)
silvia:dir5 cheeming$ cd dir2
silvia:dir2 cheeming$ svn cleanup
silvia:dir5 cheeming$ cd ..
silvia:dir5 cheeming$ svn cleanup


Still not perfect...

silvia:co1 cheeming$ svn st
A + dir5/dir2/dir4
silvia:co1 cheeming$ svn up
svn: Failed to add directory 'dir5': object of the same name already exists


So in the end, I had to remove the directory, do a revert then do another svn up and its back to normal. Very weird indeed!

In fact SVN will disallow the rename in the subdirectory if you rename the parent directory first. I didn't realise this until I found this bug. But I think its very troublesome if you want to do some major renaming of stuff.

Seems rename is already a known issue that will be "eventually" worked on but I am not sure about this directory rename issue that I am talking about. I'll have to dig deeper to find out more.

Interestingly, Mark Shuttleworth did mentioned that renaming is the killer feature for a version control system.