Code Emitter

Webseite von AdScale

Wir sind per Definition stets auf der Suche nach Vermarktungsmöglichkeiten für unsere Chat-Communtiy, so dass wir uns am 16. Januar letzten Jahres gar nicht zu arg über eine Spam-Mail ärgerten, in der uns “Deutschlands erster Echtzeit Marktplatz für Online-Werbung” nähergebracht wurde. Die Sprache ist wie erwähnt von AdScale, einer zumindest vom Prinzip her guten Alternative zu Google Adsense und Konsorten.

Nach dem inzwischen zweiten größeren ernsthaften Anlauf, AdScale als Alternative bei uns zu integrieren, möchte ich nach einem Jahr und sieben Monaten Testzeit nun ein kurzes Resumé für alle diejenigen ziehen, die ebenfalls mit dem Gedanken spielen auf AdScale zu setzen. Read the rest of this entry »

Sorry for the lack of updates recently – I have just been busy with a lot of everyday life and the following projects. Short and sweet:

Only a few days ago we started Rockz.com (“Me rockz, you rockz, we rockz.com!”) on top of our virtunity|cs plattform as a sister project for schlach.com but using a much more common name. Say hello! :)

We also launched our very own corporate website at virtunity.com – so feel free to take a look at it, too.

Bye for now! :)

Seit rund 14 Tagen läuft die Community nun bereits auf dem virtunity-System und ich muss wirklich sagen: Eine solch reibungslose Umstellung hätte ich gar nicht erwartet :-). Hat sowohl technisch wie auch logistisch dank der Mitarbeit vieler fleißiger Teammember prima geklappt.

Die Resonanz war zudem erfreulich positiv, das kann man sich so wirklich nur wünschen. Ich denke, dass wir alles in allem den Puls der Zeit und die Erwartungen unserer Nutzer auf einen gemeinsamen Nenner gebracht haben, der sich sehen lässt und absolut einzigartig im Web ist.

Es ist wahrscheinlich gerade die Klasse, die das gesamte Projekt ausmacht. Es ist keine unüberschaubare Massenveranstaltung, jedes Detail sitzt am richtigen Fleck, irgendwie kennt man jeden nur um ein paar Ecken und jeder, der etwas verbessern möchte, findet beim Team und mir immer ein offenes Ohr und ist einfach ganz nah dran. Das ist einfach “Gemeinschaft, wie sie sein sollte” :-).

Jeder, der ein wenig Zeitvertreib sucht oder sich bei den Platzhirschen zu Tode langweilt, sei hiermit natürlich auch herzlich eingeladen. Let’s rock!

Press says that in november, most big german social networks suffered from a drop in in active users. For me it seems that the initial hype is more or less over now and those people, who did not use the web for linking, tagging, sticking and other needless stuff in ugly but big social networks before the hype, got finally bored by the big barren ones and are now leaving for good.

Don’t let me lie, but if anybody out there got a fresh web 2.0 community ready to party, head for relaunch now and make this misery an end! ;-). Hah, yey, I mean us!

How can a premium sms provider offer a “payment service” while they do not garantee the actual forwarding of messages AND have no way to determine if a message has been forwarded or not? I even tried to talk to them but there seems to be absolutely no insight, that this is not a good “payment service” behaviour of a company that claims to be best in what they offer. They even told me to switch if I am not satisfied…and that no other company offers this – hah, found two! Guys, seriously, this is an absolutley basic and required functionality.

Damn, I’m sure also other customers won’t understand “these 20 lines of code are too much work for our big company to do, go ahead, you f*cking customer!” when it’s about money. Seems that they don’t have any other customer but me using their “payment services” for actual payment…hah, just lottery stuff where a single message suffices or so, I assume.

If I would do my work this way … I would not have prepared the switch already ;-)

Seems that DNS is not the cure I hoped for. I should have thought about this earlier, seems that my insight is a little bit late ;).

Explanation:

The replies that our own nameservers generate are either way reparsed by internet providers, and even with a TTL set to 0 some ISPs seem to cache anyway. Because of this is does not ultimately matter how small timeouts and what loadbalancing scheme our nameservers use -> the ISP’s caches understand round-robin only and will in some cases cache for at least 1 minute or so anyway. Of course it is possible to distribute the load to multiple servers and to implement basic failover and this is fine, but you cannot weight the different servers. For example my provider filters out all duplicate A records, so that it is not possible to do a weighting like 5xA, 2xB, 1xC. The aux config value in MyDNS is also useless because of the mentioned fact that ISP caches don’t even know the aux values and do thoughtless round-robin of the unweighted data only.

As a result of this I am trying LVS now but I am pretty sure that our hosting company will block the response packets – this is why I did not try it before. I decided to head for LVS with tunneling and I quickly set up a testing environment. Currently I am not sure if I am doing something wrong or the theory is correct, that response packets (which are sent on behalf of other physical machines) are blocked by the routers/switchports.

If all this stuff does not work, I am against my will and knowledge forced back to the http proxy/fastcgi backend SPOF stoneage cave. Depressing.

So…given the following facts….does anybody know another good hosting company where customer wishes have at least a minimal chance to become true? :-(

• HTTP proxies become ineconomic when it comes to more than X TB traffic/month per machine
• FastCGI backends suck because you are permanently fixing them (and includes point 1)
• I have no money for a double pack of high available hardware loadbalancers
• Even if I had, I have no idea where to locate them for small money (mostly because of traffic and the need to place all real servers in the same network…)
• DNS is not the cure, too (see above)
• Moreover I have no money to rent a nice cluster at Hosteurope or similar :-)
• At Hetzner it seems to be impossible to get some sort of a virtual IP or routing exception set up so that you cannot do anything even if you have the knowlege
• Apart from this Hetzner seems to be the best/keen hosting company available atm
• A fortiori I have no money for a leased dedicated line…and of course no location for this ;-)

Heh, it’s enough to drive you up the wall. Or am I just expecting too much for too few money? It could be so simple :-).

For now I’m looking for an easy way to set up DNS loadbalancing with basic failover capabilities. The first thing you find on big G is MyDNS, an ageing DNS server that serves DNS records from a MySQL database. Setup is easily done the debian way, adding records with PHPMyAdmin takes its time but is no big problem and it is also not a big deal to set up a secondary NS when simply using MySQL replication. The only thing I don’t understand is the stupid weighting algorithm. It would have been so much better to implement something that is not unpredictable. Seems to become some fun to figure this out, hah. Anyway, I already set up a primary/secondary DNS with the most important records for a bunch of zones so that I am in need of a domain hosting company that enables me to define own NS entries. We will see and I will continue on this as soon as I have learned my lessons :-).

Update: Lesson 1: in-addr.arpa SOAs seem to require some kind of mask -> 0/7.111.133.213.in-addr.arpa. works but I have no idea what this is all about. I guess: 0-255 for fixed bits in the last byte? Or is it a real netmask? Heh? ^^

Update: Lesson 2: Ok, a big IN-ADDR.ARPA. SOA suffices and holds all the reverse records. So I don’t need to think about the mask anymore :-).

Update: Lesson 3: MyDNS comes with an administration interface written in PHP. It’s broken in conjunction with PHP 5.2.0 but changing only two lines of code fixes this (change $this to something else on line 2484 and 2485). Seems to be quite nice.

Another DNS server capable of using MySQL as a backend is PowerDNS, but this seems to be just too much for my needs. There even is commercial support available what definitely turns me off. => When MyDNS does not work out, I’ll try it.

And yeah, you are right, I implemented the DNS protocol on my own on top of Apache Mina not long ago exactly for this task but as there does not seem to be any interest in neither AsyncFCGI nor AsyncDNS, there is currently no motivation left to carry on the work. However, it has been some fun and you are still free to become interested ;-).

Happy coding!

Update: Ok, I tried out MyDNS and PowerDNS now. Both of them come with everything, a good DNS server needs. In detail: PowerDNS comes with much more advanced stuff like multiple backends, master/slave configuration etc. and has got a nice documentation about every aspect of the server. However, it is more complicated than MyDNS because of this. I like the administration interface shipped with MyDNS because it is really easy to understand, while PowerAdmin for PowerDNS will get frustrating over time. However, both servers don’t really satisfy me, so I may continue my own stuff or at least create a modern admin UI for one of them when I find the time. Stay tuned! :-)

It’s time for me to drop a gear – or better – to set priorities. Yesterday I announced that I’ll discontinue most of the support work at schlach.com because the community has grown to an extend where it’s getting back-breaking fulltime work to balance the oppinions of every single user. However, there are many other experienced people using the community and they will surely answer all upcoming questions, so it’ll not make a big difference in the end.

I’m really looking forward to get back to the roots of programming and to remember how it feels to have some spare time for myself :-)

Because virtunity.com is preparing to host some bigger things soon, I had to move my blog to a brand new domain. Hope you’re still able to find it :).

Till today I used Lighttpd to distribute the load to multiple FastCGI backends. This worked “ok” until now but I’m really impressed about the performance gained by adding Varnish 2.0 (a high performance HTTP reverse proxy) in front of the web cluster. Additionally this is much easier than always fixing the FastCGI stuff when it is broken again.

However, Varnish 2.0 was released just 2 days ago, so there are still some issues. It seems that the random director (directors are used to distribute the load to multiple backend webservers) is broken currently but round-robin is working fine and should last for a while. As a workaround for using the random .weight option you can just add a backend more than once to the round-robin director.

There was also a problem with the varnishstat Munin plugin at Munin Exchange, so I uploaded a fixed version here and also added some installation instructions.

Do you wonder why Varnish seems not to cache anything?

For me (in fact I think for most people) the default varnish configuration is a bit too restrictive when it comes to cookies. All requests that contain a “Cookie” request header will not be cached – never.  As soon as a cookie is set by the site, also all static files like images, scripts and styles will no longer be cached cause the cookie is sent along with every client request, even to static files. In most situations (for almost every site that requires a login or uses adsense or similar and sets a cookie) this will render Varnish absolutely useless. A better approach is to let the dynamic pages always set a cookie, so that a “Set-Cookie” response header is created every time. Varnish also will not cache when a “Set-Cookie” response header is present so we don’t need to care about the “Cookie” request header anymore. Knowing this, all we need to do is make Varnish ignore the cookies for static files – and this is easy:

# in vcl_recv
if (req.url ~ “\.(png|gif|jpg|swf|css|js)$”) {
unset req.http.Cookie;
}

This should work for 99% of all somewhat modern websites where users can log in. Because of this I think the documentation should mention this more clearly. Also don’t forget to set etag.use-inode = “disable” in lighttpd.conf to sync the ETags when using multiple backend servers. Anyhow, Varnish is great, so have fun! :)