Code Emitter

Years ago I discovered a very comfortable way of uploading PHP files after editing them, a VBScript named “vbsFTP”. Copied into the “SendTo” folder and updated with some connection information, any file inside the base location on my harddisc could be uploaded to the corresponding remote location with just two clicks whilre preserving the local directory structure (C:\MyLocalHtdocs\a\b\c.php -> /var/www/htdocs/a/b/c.php).

However, this script is using the FTP protocol and ftp.exe integrated into Windows, so that I’m very sure, security comes short.

While installing new servers the last few days, I decided to doom the FTP servers from our brand new debian boxes and came up with my own VBScript that is using PSCP.exe (file copy over SSH, capable of private key authentication) from the Putty package instead:

Installation instructions:

1. Get the script (and PSCP if you do not have it yet)
   
2. Save the script to “C:\Documents\YourWindowsLoginName\SendTo” (XP) or “C:\Users\UserWindowsLoginName\AppData\Roaming\Microsoft\Windows\SendTo” (Vista)
3. Open it, edit the configuration section at the beginning and save
4. Make sure the local and the remote directory exist, then switch to the local directory, create or select a file, click right and select “Upload”.

That’s it! The file will be silently uploaded in the background. For creating directories you will still need to do so in the old fashioned way or you add the “-r” parameter in front of “-i” or “-pw” - but I’d not recommend that (may take a long time when accidentally clicking on a big directory). Of course you can install several copies of the script at once.

Happy coding!

I have to revise everything I said about varnish before. Meanwhile it is too hard to count the number of times, that I had to restart my instances in the last few weeks and especially today. While packet loss keeps increasing inside the Hetzner network, varnish more and more shows up as a real problem, refusing to serve images randomly with increasing frequency.

I used Varnish till today to cache images hosted on S3, but I am finally so absolutely disappointed (mostly because our users are also with us just because of varnish), that I decided to go back to good ol’ lighttpd with a proxy and caching solution I wrote in PHP (like I did before). The initial reason to switch over to varnish was, that I searched for an uncomplicated way of proxying the images to save bandwidth costs including that I do not have to touch the servers every week like I did before, but unfortunately I ended up touching them twice a day just to restart Varnish. I don’t even dare to think about the traffic peaks caused by this and all the hours I lost while logging into several servers.

I guess this is a clear case of “if you don’t do everything yourself”. However, I completely rewrote my own caching solution, made it completely independent of the backend systems and implemented a SQLite database into every instance so that it’s possible to let it clean up itself once a day via a cron job and generate some basic statistics. -> “Minimal Image Proxy 2″ was born and is already running on our image servers. It’s 3,2kb zipped.

Sigh. Finally I am really looking forward to get some peace of mind and significantly lower traffic bills from Amazon now. And it definitely feels really good again to replace troublemaking programs by a hand-crafted beautiful piece of specialiced software.

I’m currently implementing DNS on top of Mina 2. If you are interested, feel free to take a look at the makeshift page.

Maybe I’ll create a DNS based load balancing solution from this somewhen, featuring a nice Web backend for editing and adding routines for node failure detection to it (maybe Rhino based?). So, when a server is down, DNS servers will notice this and deactivate the corresponding nodes until they are back online. In combination with a good clustered filesystem this could be the base for a highly available cluster on cheap linux boxes without huge configuration needs (ok, a ~10 second downtime of single nodes still needs to be tollerated). I think I’ll look for a domain company supporting real NS entries soon :-).

Update:
Ok, I got most of the features that I personally need ready. Now you can define a datasource for DNS records and for now this can be either “Memory” (records created at runtime and fully cached) or “Database” (a JDBC layer). With the database datasource it is possible to update records on the fly while the caching features of “Memory” still remain as long as records do not change (think I’ll create a quick PHP backend for this). You could also run status daemons in the background that automatically deactivate unreachable hosts (UPDATE dns SET active=’0′ WHERE …).

You can also use every database that is supported by JDBC (e.g. MySQL, PGSQL, MSSQL, Java DB, SQLite etc.) - I think this is a big feature plus that I never saw before for a DNS server. Of course you can also implement your own custom datasources (maybe LDAP) and I am thinking about adding a simple zone file datasource so you can use the DNS server as a BIND replacement easily.

Prolog: In the last few years, I wrote PHP code mostly within Notepad. At some point I felt a strong desire for syntax highlighting and a structured source tree and therefore I am still exploring the possibilities.

As the first real need was syntax highlighting, I ended up using SciTE first. It is pretty much a more extended Notepad with the ability to highlight PHP code via a syntax highlight definition that ships with it. However, SciTE has too many different half-working highlight definitions and seems not to like wide pages so that scrolling to a far-right-away position in the middle of a line is somewhat painful. Maybe they fixed this already, but the last version I used still did so. There is also no source tree for a project folder.

Afterwards I read about Textmate, a much praised editor for Mac and I liked the screenshots/screencasts (my first from the Ruby “A blog in 15 minutes” buzz). It comes with a source tree view and pretty syntax highlighting. Unfortunately it is a commercial Mac only product and if you own one, you will pretty sure already know about it.

Not much later, E - Texteditor, which claimed to be Textmate for Windows, appeared and I gave it a try. In fact it was not hard to set it up using a really nice look and it was much fun to use it. However, there is no php documentation available directly from the editor - ok, it’s not an IDE, it’s an editor. In the first releases there were some bugs and there still seem some to be left especially when quickly closing and reopening E, but I can recommend this editor to everyone who has $ 34,95, wants a very pretty editor and currently misses syntax highlighting and a source view. Imho E is the best Notepad replacement for editing PHP code available on Windows and it can also do much more.

On Linux, gedit is an excelent tool that also ships with gnome. It’s basically the same like E but does not look so pretty. However, editing PHP files with it does just work and there has been some progress recently to make PHP highlighting more comfortable. You may need to tweak it a little to fit your needs, but because you are using Linux, I assume you know where to find more information on this :P.

Not long ago I also noticed Netbeans’ approach to PHP editing and because I have been using Netbeans for a long time to write Java already, it was a must to try it out. At the first look it is just good old Netbeans as you know it from Java but the Early Access for PHP edition features PHP only instead including code completition and so on. Unfortunately creating a new project from existing source folders is not yet trivial and there are some settings that you will never need without appserver integration. However it is possible to create a new project just by using the default settings, steal the “nbproject” directory from it, copy it to the root folder of your PHP app and edit the “project.properties” and “project.xml” to fit your needs. For example:

C:\yourapp\nbproject\project.properties:
copy.src.files=false
copy.src.target=
include.path=\
    ${php.global.include.path}
index.file=index.php
source.encoding=UTF-8
src.dir=C:\\yourapp
url=http://localhost/yourapp_path_maybe

C:\yourapp\nbproject\project.xml:
<?xml version=”1.0″ encoding=”UTF-8″?>
<project xmlns=”http://www.netbeans.org/ns/project/1″>
    <type>org.netbeans.modules.php.project</type>
    <configuration>
        <data xmlns=”http://www.netbeans.org/ns/php-project/1″>
            <name>YourappName</name>
        </data>
    </configuration>
</project>

Done so, when opening the project (just the root folder of your app) you are presented with your PHP application loaded inside Netbeans with all your files in the source tree view. From now on you can edit your project easily and, if you setup a local webserver (take a look at Server2Go or Lighty2Go if you want a small but just working local webserver/mysql/php stack), get live results. For serious PHP programming (yes, not everyone acknoledges this ^^) the free Netbeans for PHP is a nice tool. I am looking forward to see the next releases. Till then I will use it for a while and hope that it will become simpler to use in the future without all the bloat :).

There is also a PHP plugin available for integration with Eclipse (this is also official supported by Zend) but because I never were the typical Eclipse guy (don’t know, just not the way I like things to look / be done), I did not test this in much detail. As far as I read the Eclipse PDT will integrate somewhat deeper with PHP thus featuring a debugger and easy integration with the Zend Framework. As I’m still a Notepad-Era guy running his own framework code, this may not be exactly what I am looking for (I also complained about the somewhat forced appserver integration in netbeans which is quite the same, you remember). However, if you plan to use most of the features of a complete IDE supported by the Zend guys, Eclipse PDT will be your way to go currently.

<?php die(”So far…”); ?>

Jemand hier, der aktuell das Verteilte Systeme-Praktikum besucht oder es noch vor sich hat? Falls ja, kleiner Exkurs:

Wir kennen ja alle das altbewährte Framework inkl. Network Simulator, das innerhalb des Praktikums eingesetzt wird und tausende Fenster auf dem Bildschirm ungeordnen öffnet - so ziemlich für jedes Objekt zwei Stück. Grundsätzlich ist diese Lösung für wenige Komponenten und all diejenigen in Ordnung, die sich mehr in C als in Java beheimatet fühlen. Beim letzten Praktikum (vspra5) wird es allerdings ziemlich grausam, z.B. drei Clients, einen RegistryServer, drei Zeitserver und den Simulator zu überblicken - wohl auch aus dem Grund, dass man bei jedem Durchlauf alle Fenster neu positionieren muss und auch kaum die Übersicht behalten kann, welches Objekt mit welchem vs_comm etc. Objekt kommuniziert, welches Ausgabefenster verwendet und und und…

Habe mich der Problematik daher einmal angenommen und ein Framework speziell für den Einsatz im letzten Praktikum erstellt, da dieses ja ziemlich viel Freiraum lässt. Herausgekommen ist aus Studenten-Sicht ein einfaches Client-Server-Modell, das automatisch den integrierten Simulator nutzt und in einem einzigen Fenster mit strukturierten Tabs für Clients, Server und anderes (z.B. der Simulator selbst) läuft. Auf Basis des Frameworks ist es sowohl für diejenigen mit Java-Erfahrung als auch für nicht-C-Programmierer, die sich lieber mit Java als mit der reinen Aneinanderreihung von Funktionsaufrufen auseinandersetzen möchten, ein Leichtes, die entsprechenden Server und Client zu implementieren. Einen kleinen Screenshot hätte ich auch anzubieten:

Das Framework besteht aus drei kleinen Paketen:

vs.common beinhaltet den Simulator und alle übrigen notwendigen Klassen, mit denen man sich selbst bei der Implementierung nicht mehr herumschlagen muss - der Simulator bleibt vollständig im Hintergrund. Clients und Server implementieren eine abstrakte Komponente, die jeweils ein entsprechendes Tab im Hauptfenster anlegt und zusätzlich Funktionalitäten wie den Einfachen Zugriff aufs Ausgabefenster wie auch auf die Eingabezeile (für eigene Befehle) mit sich bringt - muss man sich aber auch nicht mit auseinandersetzen, ist einfach verfügbar direkt in den eigenen Objekten. Grundsätzlich ermöglicht das Ganze den Umgang mit dem Simulator, als wäre er ein echtes Netzwerk. Dazu erlaubt der Simulator ebenfalls die Umkonfiguration des Netzwerkes direkt über die entsprechende Befehlszeile (die Befehle werden beim Start entsprechend aufgelistet).

vs.client beinhaltet nur einen Client, der die Methoden println(String text), send(DatagramPacket packet), receive() und receive(int timeout) implementiert. Ein einfaches “TimeClient extends Client” und das Hinzufügen der entsprechenden knappen Logik genügen - die Beschäftigung mit Kommunikationsobjekten, das Zuweisen von Ausgabefenstern etc. entfällt vollständig, da alles automatisch vom Framework erstellt und zugewiesen wird.

vs.server beinhaltet analog einen abstrakten Server, der es erlaubt, die Methoden onReceive(DatagramPacket packet) und onTick() hinzuzufügen, um die entsprechende Programmlogik für einen TimeServer oder RegistryServer zu implementieren (RegistryServer extends Server etc.). Ebenfalls verfügbar sind Methoden zum einfachen Starten und Stoppen eines Servers sowohl die implementierbaren Methoden onStart() und onStop(), z.B. für die Registrierung der TimeServer beim RegistryServer, immer wenn ein Server startet.

Sowohl Clients als auch Server können zudem die Methode onCommand(String command, String[] args) überschreiben, um eigene Befehle aus der Befehlszeile abzugreifen - ist also vollständig modular, man braucht keine Knöpfe und regelt alles in einem einzigen Fenster.

Falls jemand Interesse hat, einfach melden…und wenn nicht, trotzdem viel Spaß ;-).

Not long ago, Sun released a bundled package of their “Sun Java System Application Server” (Glassfish) and MySQL. I took this as the reason to try out PHP on Glassfish with enabled MySQL support. The initial installation was quite simple but there were some problems with documentation about the usage of MySQL. For those of you who already have a MySQL server running and just want ot test your application on Glassfish, the installation basically is:

1. Download Glassfish/MySQL Bundle
   Or if you already have glassfish, add the Mysql Connector/J (the jar from the archive) to GLASSFISH/lib
2. Download Quercus (most recent) and unpack the following files from the archives:
   quercus.jar
   resin-util.jar           to       GLASSFISH/domains/domain1/lib
   script-10.jar
3. Add the following to GLASSFISH/domains/domain1/config/default-web.xml:
     <servlet>
       <servlet-name>Quercus Servlet</servlet-name>
       <servlet-class>com.caucho.quercus.servlet.QuercusServlet</servlet-class>
       <init-param>
         <param-name>script-encoding</param-name>
         <param-value>UTF-8</param-value>
       </init-param>
       <init-param>
         <param-name>database</param-name>
         <param-value>ExampleMySQL</param-value><!– See below –>
       </init-param>
     </servlet>
     <servlet-mapping>
       <servlet-name>Quercus Servlet</servlet-name>
       <servlet-class>com.caucho.quercus.servlet.QuercusServlet</servlet-class>
       <url-pattern>*.php</url-pattern>
       <init>
         <!– <php-ini>
           <sendmail_from>my_email_address</sendmail_from>
           <smtp_username>my_email_username</smtp_username>
           <smtp_password>my_email_password</smtp_password>
         </php-ini> –>
       </init>
     </servlet-mapping>
4. Launch Glassfish and log in to the administration interface (by default: http://localhost:4848)
   Howto: Glassfish Quickstart Quide
5. On Resources / JDBC / … in the administration interface add a new connection pool of type MySQL and set User/Password/Port/URI etc. in its properties (look through them when creating and don’t leave them blank). When done, “ping” (the button) your MySQL server from the connection pool view to test if it worked.
   Also add a new “JDBC Resource” that links to the pool and give it a name, in this case ExampleMySQL.
6. Add a PHP file of your choice to GLASSFISH/domains/domain1/docroot and maybe restart Glassfish once to be sure the database changes are live.
7. Run your MySQL-enabled PHP script (e.g. http://localhost:8080/index.php).

That’s basically everything. It took me some time to find out about the correct MySQL configuration (it seems that other tutorials I discovered at Google are quite outdated). Now, go and have fun!

Errrm: There are also enterprise clustering features implemented into Glassfish. I am wondering if this can scale a huge PHP site as well as lighttpd/php-fcgi. If anyone knows more details about this, please let me know. And mh: Just a thought: What about Glassfish + Quercus + MySQL + GridGain to scale at least the Webserver part into new dimensions? ;-)

One of my recent activities focused on the excelent Apache Mina NIO framework. I played around with it while creating an improved version of the current chat server for Schlach.com and besides added FastCGI support to Mina to try out the possibilities. With the library it is now possible to develop efficient FastCGI server applications easily and I also added all the tools to talk to remote FastCGI servers like PHP/Ruby etc.

If you are interested in trying this out, take a look on its makeshift page. The client side code seems to be still a little slow, so if you find some improvements especially to using the IoConnector, please let me know.