My RSS » Items by Alan Knowles

Items by Alan Knowles

Planet PHP

app.Builder.js - First release

Posted: May 25th, 2010, 10:03am MDT by Alan Knowles
Article originally from AK BK Consulting blog
A Release!!! - Well Idle hands make cool stuff...
What is It?Basically Glade on steriods. (and it needs a good name!!)
What?Well Glade the UI builder is great for what it does, however app.Builder.js attempts to solve a few roadblocks in Glade
- Moving stuff around is klumsy - no drag drop - It's not very good for prototyping layouts...
- It's limited in what widgets can be added
- Coding (when used as an Ajunta addon) is just downright messy.
- It's difficult to extend/modify (and in some cases next to impossible)
- Adding additional properties to widgets is difficult
- Information is often hidden from display (how many clicks to find out what signals are connected)
- There is no 'run it and see' option.
So what makes this different..?
- It's all done in Javascript - using a tree like structure (and it's fully self hosting) - the application is used to modify it'self.
- It uses GObject Introspection. which solves a number of issues
- Making extra widgets to the application is just a matter of editing files (eventually a UI will enable you to pick from a list of available Introspection elements... )
- You can build web Applications (using toolkits like RooJS) - ExtJS, Moo etc. could be easily added..
- It should be quite easy to build Clutter applications...
Do I need to know anything to use it.Yes, one of the core concepts behind the builder is understanding XObject, a simple Javascript wrapper around GObjects
that enables you to create a JSON like tree, and XObject will turn your tree into an application.

Key features of the XObject are
- xtype - the name of the GObject (eg. Gtk.Window)
- packing - the pack property, can be a string (comma seperated indicating how an element is packed to it's parent)
- id - all objects can have ID's you can fetch any object using this.get('the_id') as long as it's a child of the current element. To look up the tree, use a '.' at the start, or to start at the top of the tree use '/', eg. this.get('/Window') get's you the window. this.get('/Window.toolbar') get's you the first toolbar in the window.
- el - when an element is created, the GObject is always stored in this.el
- |pipe - properties - currently the editor uses '|' as a prefix to indicate the value is raw Javascript (eg. a function or Value, that when used should not be quoted as if it was a string.
What state is it in?Well, It's self editing.. - The application is used to create it's self, just create a new Gtk Project, and point it at the Sample directory, and you can edit the current codebase. (copying it to the Builder directory when you have a stable build)
What's to do...Quite a few tweaks.. It works, but could do with more time in polishing the interface. My personal goal is to save time using it as a productivity tool, for development. Which is basically where it's at..
Where do I get it?Download a tarball app.Builder.js.2010-05-25.tgz or git clone [git.akbkhome.com]

once unpacked/cloned.run install_gir.sh - to update your gir files.If you want to try RooJS editor - you need git thenrun update.sh
What do I need to get it to work?basically a working gobject instrospection installation (.gir files and .typelib files) and the latest version of the Gnome seed interpreter- I know it works on Linux. I would be interested to hear if it works on other platforms.Note: there is a update script in the distribution which fixes the currently available Gtk gir's (please read the readme.txt for more info)
Application Builder slowly growing

Posted: May 13th, 2010, 9:16pm MDT by Alan Knowles

Article originally from AK BK Consulting blog

While paid work is still horribly quiet, my little application builder is getting closer to usable.
I posted a video a while back showing the web version of the Application builder, this is the next generation, a desktop version all done in Javascript using Seed. Not only can it build Roo applications, but also Gtk ones.. (and in theory maybe JQuery/Prototype etc..)
The main reason to switch to a desktop version was the addition of GtkSourceView, which will enable autocompletion and proper code editing.

It's really a proof of concept, but I can easily move it out to github from it's current home - [git.akbkhome.com] if anyone want's to help out.
download:
git clone [git.akbkhome.com]
run update.sh once download to get the roo library
Application Builder slowly growing

Posted: May 13th, 2010, 8:52pm MDT by Alan Knowles

Article originally from AK BK Consulting blog

While paid work is still horribly quiet, my little application builder is getting closer to usable.
I posted a video a while back showing the web version of the Application builder, this is the next generation, a desktop version all done in Javascript using Seed. Not only can it build Roo applications, but also Gtk ones.. (and in theory maybe JQuery/Prototype etc..)
The main reason to switch to a desktop version was the addition of GtkSourceView, which will enable autocompletion and proper code editing.

It's really a proof of concept, but I can easily move it out to github from it's current home - [git.akbkhome.com] if anyone want's to help out.
Inotify and Git auto commit then push

Posted: April 8th, 2010, 1:55am MDT by Alan Knowles
After hacking on a few gnome libraries, seed, gobject-introspection and filing a few bug reports with patches, I've been slowly getting used to git.

Previously I have used subversion for all my VCS requirements, It has proved very useful over the years, saving my ass a few times, and generally just easy to deal with. But as Im doing quite a bit of experimental hacking on gobject-introspection, the concept of keeping a huge patch file up-to-date was becoming a serious pain, along with making it impossible to submit to the maintainers.

So given that my understanding of Git, was that it was designed to solve this specific problem (you working on a experimental version of a public codebase). I though I would investigate further. Coming from a CVS/SVN background, my first thought was that I should ask the maintainers to let me create a branch on the public git for this. - This is normally what you would have to do if you where using the centralized CVS/SVN repositories. But the more I read and understood, it became clear that rather than having a centralized branch, you just have to set up your own public git repository copy (and in my case put it on my server, as I work from a multitude of workstations and a laptop). Then basically you can hack away at your hearts content. making as much mess as you want, and when you are ready, tidy up your own commits into a series of 'proposal' commits and suggest they pull those change from your server. Meanwhile you can keep your changes intact, while still syncing with updates they do. An absolute masterpiece in design there...

The only part I've found is lacking is that the generation of this 'quality patch' is a bit clunky. Basically after a while, you find that you need to merge a number of commits from your own repository to generate this patch. I've got about halfway through writing a little Gui Patch creator by reordering and merging commits together, as I could not find a tool that made this as simple as

"Select Branch point, Join/break apart patches, re-order and then create a branch with a series of tidy patches with nice commit message."
(well that's for another post...)

Anyway the second way I use subversion at present is basically webdav mounted on all my workstations, when I save the file, it is written via webdav to subversion. which in turn using some simple hooks writes it to a live area, where I can test stuff (eg. websites etc.)

This method of working is extremely efficient, and pretty safe (I can undo deletes, etc.), it also provides a handy way of cost estimation for billing (by looking at the commit times etc.). The only serious downside is that it's highly dependent on my internet or wifi connections being stable and fast. The later quite often being a serious problem, especially if the editor wants to open all the files in the project to determine auto-completion data. Essentially I've had to turn off some of these features for the sake of usability.

I had being wondering about an alternative for a while, initially thinking of a similar webdav based arrangement. However after toying with git for quite a while I finally worked out a far better solution. So here it goes.

On the server side - I have git running with the new simple http cgi (as webdav based git DOES NOT WORK!) - however much the document hints - you can not commit to a webdav mounted git repository, as it will never run any of the necessary hooks. The simple http cgi is also a bit flakey if you do not have git 1.7 on both ends.

This has a very simple post-update hook that basically updates a live copy of the repository (in my case /home/gitlive/{repository} ) Thanks to [joemaller.com]
```
#!/bin/sh
cd /home/git/myrepository || exit
unset GIT_DIR
git pull
```
Now on the desktop end, is where the clever stuff comes in. I create a directory ~/gitlive and checkout a working directory in there
```
git clone [mysite] 
```
I then download this little script
```
curl [git.akbkhome.com] > gitlive.js
```
then run it by entering on the desktop F2(run)
```
seed ~/gitlive/gitlive.js
```
(run it from the terminal to see what's going on if nothing works.)

What this does is

a) use inotify via g_file_monitor to watch everything in ~/gitlive

b) uses the Desktop Notification API to tell you what its watching

c) put's a little icon in the system tray (so you can quit it)

d) every time i
Truncated by Planet PHP, read more at the original (another 913 bytes)
Seed doc updates, and Gio async directory listing

Posted: March 7th, 2010, 10:52pm MST by Alan Knowles
The documentation for seed gobject introspection is improving continually, I now have a jhbuild virtual machine, which is picking up the latest versions from git.
In addition many of the documentation details have been expanded, including
- Interfaces, Enums
- callback methods are now documented
- More libraries have been added.
- More doc comments can be picked up
With better documentation it's finally possible to figure out how to use the API. A classic example of this was a small problem I tried to solve before the API documentation was available. Listing a directory asynchronously, It took me over an hour to get close to figuring out how to do this, I eventually had to give up. as digging through the source, C reference and GIR files took so long.
However within 5 minutes with the documentation, I was able to write a small script to do this.
```
Gio = imports.gi.Gio;
Gtk = imports.gi.Gtk;

var f = Gio.file_new_for_path('/home/');
f.enumerate_children_async (
        "*",  
        Gio.FileQueryInfoFlags.NONE, 
        GLib.PRIORITY_DEFAULT, 
        null, 
        function(o,ar) {
          // listing completed..
           var fe = f.enumerate_children_finish(ar);
           var ch = false;
           while (ch = fe.next_file(null)) {
               Seed.print(ch.get_name());        
           }
           Seed.quit();
    
        },
        null);

Gtk.main();
```
Reported Attack Site - recovering from gumblars

Posted: March 3rd, 2010, 1:58am MST by Alan Knowles
Just about finished a gumblar cleanup, for a small Hong Kong company. This is not the first crack I've seen in the last few months, I fixed another server last month that got ssh brute force attacked. It looks like cracking is on the up, so if you need help fixing a site, by someone who knows what they are doing, and at the same time you will help out a number of open source projects - give me a bell (alan@akbkhome.com)

The gumblar (or derivative) attack I was looking at was quite interesting, the first indication the owner got was that browsers kept showing the "Reported Attack Site!" or "Warning: Visiting this site may harm your computer" message. So I get the call to find out what's going on.

When you ignore the message and go through to the site, look at the html the first thing you see is that there is a <script> tag added just before the body pointing to a gifimg.php file. After that you have a long hunt around google to find out what's going on.

At the time of writing, the exact attack vector does not look like it's been confirmed, but is either a brute force ftp attack (I think is quite unlikely considering the username/pass combo on this sample site). Or more likely a PDF desktop attack to a machine that has access to the site.

My first assumption was that it was a Wordpress exploit, but the more I examined the situation, it seemed less likely. However I highly suspect that the PDF attack vector having got the ftp credentials goes looking for standard locations of wordpress installations (eg. '/wordpress) - so hint one is not to install your software in such obvious places.

Cleaning it out
The first step in sorting out the mess was to mirror the original site, with virus and all onto a offline location. (both as a precaution that if we broke things we had a backup, and so we can use this as a source to replace the hacked files with new ones).

After that it was a matter of googling for details of the attack and writing a gumblar cleaner script. It basically checks for infected file types, then preg_replaces out the hacked additions. These include
- php files with an eval/base64_encode line
- javascript files with document.write lines
- html, shtml and htm files with <script tags.
I used ftpput, and check return values, to ensure that each file was successfully replaced before overwriting the local copy and making a nice copy for my reference into the virus folder.

Inside out of the attack.
The infection is quite interesting, and in this case was quite painful, due to the nature of how Wordpress publishes files.

Initially I suspect the core code in the PDF actually has some ftp code which will try and modify standard set of PHP files to add a small base64_encode script.. (phplist, and wordpress appear to be core targets, and I'm sure there are more.)

This is a snippet of some of the code that get's added (it's all eval, base64_encoded - read up on my blog post about idiot ways to protect your PHP code using this idea.)
This is a snippet of the decoded script
```
if(!function_exists('kqyf')){
   function kqyf($s){

     ... infect the page stuff goes here...  
	}
	function kqyf2($a,$b,$c,$d){
		global$kqyf1;
		$s=array();
		if(function_exists($kqyf1))
			call_user_func($kqyf1,$a,$b,$c,$d);
		foreach(@ob_get_status(1)as$v)
			if(($a=$v['name'])=='kqyf')
				return;
			elseif($a=='ob_gzhandler')
				break;
			else
				$s[]=array($a=='default output handler'?false:$a);
		for($i=count($s)-1;$i>=0;$i--){
			$s[$i][1]=ob_get_contents();
			ob_end_clean();
		}
		ob_start('kqyf');
		for($i=0;$i<count($s);$i++){
			ob_start($s[$i][0]);
			echo $s[$i][1];
		}
	}
}

$kqyfl=(($a=@set_error
```
Truncated by Planet PHP, read more at the original (another 1041 bytes)
Reported Attack Site - recovering from gumbars

Posted: March 2nd, 2010, 2:25am MST by Alan Knowles
Just about finished a gumbar cleanup, for a small Hong Kong company. This is not the first crack I've seen in the last few months, I fixed another server last month that got ssh brute force attacked. It looks like cracking is on the up, so if you need help fixing a site, by someone who knows what they are doing, and at the same time you will help out a number of open source projects - give me a bell (alan@akbkhome.com)

The gumbar (or derivative) attack I was looking at was quite interesting, the first indication the owner got was that browsers kept showing the "Reported Attack Site!" or "Warning: Visiting this site may harm your computer" message. So I get the call to find out what's going on.

When you ignore the message and go through to the site, look at the html the first thing you see is that there is a <script> tag added just before the body pointing to a gifimg.php file. After that you have a long hunt around google to find out what's going on.

At the time of writing, the exact attack vector does not look like it's been confirmed, but is either a brute force ftp attack (I think is quite unlikely considering the username/pass combo on this sample site). Or more likely a PDF desktop attack to a machine that has access to the site.

My first assumption was that it was a Wordpress exploit, but the more I examined the situation, it seemed less likely. However I highly suspect that the PDF attack vector having got the ftp credentials goes looking for standard locations of wordpress installations (eg. '/wordpress) - so hint one is not to install your software in such obvious places.

Cleaning it out
The first step in sorting out the mess was to mirror the original site, with virus and all onto a offline location. (both as a precaution that if we broke things we had a backup, and so we can use this as a source to replace the hacked files with new ones).

After that it was a matter of googling for details of the attack and writing a gumbar cleaner script. It basically checks for infected file types, then preg_replaces out the hacked additions. These include
- php files with an eval/base64_encode line
- javascript files with document.write lines
- html, shtml and htm files with <script tags.
I used ftpput, and check return values, to ensure that each file was successfully replaced before overwriting the local copy and making a nice copy for my reference into the virus folder.

Inside out of the attack.
The infection is quite interesting, and in this case was quite painful, due to the nature of how Wordpress publishes files.

Initially I suspect the core code in the PDF actually has some ftp code which will try and modify standard set of PHP files to add a small base64_encode script.. (phplist, and wordpress appear to be core targets, and I'm sure there are more.)

This is a snippet of some of the code that get's added (it's all eval, base64_encoded - read up on my blog post about idiot ways to protect your PHP code using this idea.)
This is a snippet of the decoded script
```
if(!function_exists('kqyf')){
   function kqyf($s){

     ... infect the page stuff goes here...  
	}
	function kqyf2($a,$b,$c,$d){
		global$kqyf1;
		$s=array();
		if(function_exists($kqyf1))
			call_user_func($kqyf1,$a,$b,$c,$d);
		foreach(@ob_get_status(1)as$v)
			if(($a=$v['name'])=='kqyf')
				return;
			elseif($a=='ob_gzhandler')
				break;
			else
				$s[]=array($a=='default output handler'?false:$a);
		for($i=count($s)-1;$i>=0;$i--){
			$s[$i][1]=ob_get_contents();
			ob_end_clean();
		}
		ob_start('kqyf');
		for($i=0;$i<count($s);$i++){
			ob_start($s[$i][0]);
			echo $s[$i][1];
		}
	}
}

$kqyfl=(($a=@set_error_ha
```
Truncated by Planet PHP, read more at the original (another 1038 bytes)
Generating Seed Documentation from Gobject introspection

Posted: February 25th, 2010, 8:18am MST by Alan Knowles

Job hunting is going pretty slowly (if anyone at ubuntu is listening hint hint ;) , and projects are pretty much done, so I get to work on really cool stuff.
While developing in seed is pretty productive, hunting down the documentation, or discovering the right method to use has been one of the roadblocks I have been bumping up against. So wondered yesterday how well the introspection code in seed works.
Turns out, that the built in Seed.introspection() method is pretty pointless, even after committing a few extra features, I discovered it was never going to be much help. However girepostory has it's on gir/typelib file, so you can call introspection directly.
After a few experiments, I worked out to inspect basically every method, and class in Gnome. Hence there is now full documentation for every Gnome Class that seed can talk to (this does not include libxml however, as it does not use gir to talk to that.
Support for doc comments (by parsing the gir files) is there, but the machines I build on unfortunately have rather old gir files, which do not include all the docs comments.
This code could be very easily refactored to generate docs for pygi, gjs or vala.. or any other gir based binding..
For your viewing pleasure..
Seed Gnome Documentation
If it's not working, it's probably as I'm working on it and have broken it.
The code is in my seed subversion repo the key files that do all the work are JSDOC/Introspection.js, and docs.js The rest of the code just deals with Template parsing and handling (/docs/* are the templates)
To run it you need cutting edge seed (which can take quite a while to set up as it depends on a recent webkit build)
Enjoy.
Drag and Drop with seed (Javascript) and Gtk

Posted: February 16th, 2010, 8:10pm MST by Alan Knowles

As part of my, "What to do while looking for a Job" project, I'm exploring the Gtk/Gobject introspection bindings in seed.
It's one of those fun, not documented anywhere, and you have to dig around all the source to understand how to do things projects. And part of a grander plan to develop a generic application builder loosely based around the RooJs builder application I blogged about previously.
The idea is that using Gtk, GtkSourceView, and the concepts I developed in the RooJS Builder, I would be able to create Dynamic Web applications (and eventually Gtk ones) very rapidly due to the ability to closely tie the action code to the interface design.
The core reasons why it speeds up development are:- Syntax checking is feasible when creating code.- Adding, editing and selecting properties and event handlers is faster, less error prone, and access to documentation is instant. - Visually changing and seeing the effect of a change (the change - reload - test cycle) is reduced to milliseconds rather than tens of seconds.
So far most of the application is mocked up, (and relivant patches have been submitted to seeds' bugzilla for review). However one of the key components I was looking at over the Chinese New Year Break was drag and drop of Palete Items onto the Rendered View of the application (Webkit embedded), (and eventually the tree that makes it up). To do this involved seriously testing the Gobject introspection bindings and how the interact with the drag and drop methods in Gtk.
So Read on if you are interested in how it all comes together.
Javascript on the Desktop (well Linux Mostly)

Posted: January 19th, 2010, 2:36am MST by Alan Knowles
Unfortunately my internet line died over the weekend, and I was left without a connection until Tuesday. I had a bit of offline time to have a look at some interesting new(ish) technology.

If you have paid any attention to Gnome development, there are mentions of gnome-shell, and something about making it easier to develop add-on/applets etc. using Javascript. I have not really had time to look at this much, however given the fact I had downloaded all the components prior to my loss of connectivity, I spent some time over the weekend and monday looking at this in more detail.

It looks like there are obviously things going on in IRC and mailing lists, so most of my impressions are from the websites, and code. There are basically 2 projects currently.

GJS - This is the mozilla Javascript engine bound to the gobject introspection system
Seed - This is the webkit Javascript engine bound to the gobject introspection system

Of the two, Seed apears to be a little more advanced and more time has gone into it, based on the fact there appear to be more core features in Seed (eg. I could not find anything like Seed.print() in GJS), and there are very few examples in GJS

Both of these use the GObject introspection method to bind to Gtk, and a large and growing number of other libraries. This is something I found very interesting having gone through a similar process with rooscript.

The fundimental issue of creating bindings to Gtk (and anything that uses the GObject system) is that historically, almost everyone (PHP, Python etc.) who did it used this method.

* Parse the html documentation, or the .h files (extracting the classes, enums, methods, arguments etc.)
* Generate Binding code for each library, which wraps these methods, loads the '.so' libraries and links it all together.
* Compile a Module (or statically link it in some cases)
* Run... Test.. Fix bugs in wrapping code... Run ... Test.

What GObject inspection introduces is an very thin layer that can be used to expose any of these methods without writing any binding code (other than to GObject inspection). The result is that to add more features (linking to another library) - you just have to generate a XML file describing the interface, compile it with g-ir-compile and put it in the right folder, and you have magically added support to a new Library, without generating an C code!

This also means that fixing the binding is considerably simpler (fix the XML file, re-compile etc.) and you have solved bugs in any language that is using it... (as I found with gtk_tree_store_set_column_types()

Playing with the languages.
GJS and Seed present the Gtk API very slightly differently, which at present appears to be a bit of a blocker for deciding which to use (or even to bother for some). The key differences are for authoring

* Javascript 'let' features in GJS - not available in Seed.. - This is rather a big blocker as code designed for GJS will fail in Seed totally.. (and visa-versa probably)
* Different call signatures to signalsÃï¿½ÃÂ Ãï¿½ÃÂ Ãï¿½ÃÂ SEED:OBJECT.signal.connect(method)Ãï¿½ÃÂ vsÃï¿½ÃÂ GJS:OBJECT.connect('signal', method)Ãï¿½ÃÂ - I think personally that GJS way is more true to the concept and cleaner, however as you can see below there are ways around this.

Seed has a git repo on gnome 'seed-examples', which contains a large number of examples, although their depth is rather thin unfortunatly.. (code coverage is probably 1% at best)...
Both have pretty much Zero in the documentation stakes.. - A rather trivial task to create - which I'm tempted to have a go at....

Anyway back to my playing around..
I've been doing some serious shit in Javascript recently, GUI builders, Code generators, Hacking GTK bindings etc. so I've seen the good, bad and ugly as far as Javascript goes (Bad sometimes being my code).Ãï¿½ÃÂ I've been slowly moving to what I regard as an effecient, productive use of Javascript, most of which can be seen in the xtype support in RooJS

After runing the first few examples in seed-examples, I decided to see if using an xtype / Roo structure could be done with Seed. After a bit of tweaking, and bugfixing of Seed, I finally got this code to work.
```
var win = XN.xnew({
    xtype : Gtk.Window,
    type: Gtk.WindowType.TOPLEVEL,
    listeners : {
        'delete-event' : function (widget, event) {
            return false;
        },
        destroy  : function (widget) {
            Gtk.main_quit();
        }
   },

   set : {
        set_border_width : [ 10 ],
        resize : [300, 300],
        show_all : []
   },
   items : [
```
Truncated by Planet PHP, read more at the original (another 5487 bytes)
RooJs Appbuilder preview

Posted: November 14th, 2009, 12:30am MST by Alan Knowles

Well, a peek into the secret world of Roo development. I've been a bit frustrated with the speed of development of the applications recently, so I decided to speed it up a bit..

Below is a little video of a module I've added to the Iconstruction framework (which might get released one day..) - which enables me to very rapidly build applications.

The knock on effect of developing the builder has been that there is now far better support for xtype object constructors in Roo, along with far more acurate and complete documenation (as the builder tool needs that to work)
Ubuntu release 9.10 a bit early

Posted: October 29th, 2009, 5:46pm MDT by Alan Knowles

Looks like ubuntu released karmic today (as I guess they could not rename it 9.11).
I've been testing karmic for months, (which at times have been very painful - but if you join the beta ride, what else would it be), but in the last few weeks it looked like it was really coming together.
The only big regression was the synaptics touchpad resetting the 'tap' setting after suspend resume cycle.
However, there was a kernel update last week which completely broke suspend (it never really sleeps the machine), and attempts to report it via the apport tool result in a invalid message about the kernel not being a ubuntu one.. (which someone has reported)
I'd really suggest anyone requiring resume/suspend to work, to hold of for a little while until they sort out the remaining issues.

Speedier google translate API for RooJs and ExtJS

Posted: October 13th, 2009, 5:01pm MDT by Alan Knowles

If you have ever used googles translate API, it can quickly become a love hate relationship. You love the features it provides, but you begin to hate that fact that google's server are slow and flaky for loading the libraries that they recommend.

This situation was getting especially annoying yesterday, as the load time of my application (that is getting continually re-loaded while I'm working on it) was getting worse and worse, and google's API's was the culprit.

So after some further reading on that page, I realized that the translation call was really just a simple HTTP request with the correct parameters.. no need for huge google framework API etc.

So here's the ~20 line javascript to replace the slow loading 50k+ library that google recommends....

/**
 * usage 
 * gtranslate('hello', 'en', 'es', function (res) {
 *     if (typeof(res) == 'object') { return; } // failure
 *     console.log(res); // success...
 * });

*/

function gtranslate(str, src, dest, cb) 
{
        var x = new Roo.data.ScriptTagProxy({ 
            url:  'http://ajax.googleapis.com/ajax/services/language/translate', 
            callbackParam : 'callback' 
        });
        x.load(
            {
                v: '1.0',
                q : str,
                langpair : src + '|' +dest,
            }, // end params.
            { // reader
                readRecords : function (o) {
                    if (!o.responseData) {
                        return o;
                    }
                    return o.responseData.translatedText;
                }
            }, 
            function (result) {
                cb(result);
            },
            this,
            []
        );
            
        
    }

PEAR state of play, why move to PEAR2

Posted: September 21st, 2009, 12:16am MDT by Alan Knowles

Just before I saw Brendan's post about PHP4 compatibility in PEAR, I had been getting a few queries about making a couple of my PEAR packages more 'PHP5' compatible or PEAR2 ready.

From my perspective, pretty much all of the packages I maintain (As far as I know) are PHP5 'compatible'. however they may emit E_STRICT errors.

This brings up the interesting question, which I guess all the current maintainers, users and contributors have come across, how much value is added to the existing packages by adding that level of support.

From an 'ideal world' / 'perfect coding' perspective, they would benefit from this changes. but as somebody who earns an income by delivering projects as quickly and efficiently as possible, the return on investment for making those changes is very tiny, if not negative.

Since the packages generally just work, making the changes required, would not really change that 'just work' situation, and as Jamie Zawinski famously said "How will this software get him laid?"

Two of the biggest changes I'm aware of for this 'PHP5 compatibility' issue are the 'static' method prefix and getting rid of 'var' which completely break PHP4 compatibility (and yes we still maintain PHP4 applications, and clients rarely have budget to make changes like this). Doing these changes would mean that I would have to either freeze or depreciate PHP4 support, or start maintaining dual versions. (Personally I would prefer a hook in the package builder that would do the replacement for me, so I could upload 2 package on each release).

Going forward, PEAR2 is still in a gestation period, (as PHP5.3 and namespaces support has just come out.) Resulting in any code that had targeted PHP5.3/PEAR2 aging very quickly (eg. requiring changes to handle the final changes to the namespace syntax.). This may start changing soon, however I suspect it would really take some significant effort in time to start creating PEAR2 packages for existing code (which has a rather poor return on investment) . And without a reasonable base number of packages, the attraction of submitting code to PEAR2 is lessened. A classic chicken and egg situation.

At the same time, there is no real alternative to PEAR2, pretty much all other 'framework' solutions have been built around the assumption that you have to accept a majority of the 'framework' to utilize the single packages. Which is even worse that the pains that PEAR(1) imposes on you.

All that said, if you want to send me patches to fix any big PHP5 issues in my packages please don't hesitate, I will try and make the changes.
Barcamp Hong Kong 2009

Posted: September 13th, 2009, 4:39pm MDT by Alan Knowles

Went along to Barcamp in Hong Kong this weekend, Have to thank all the guys & girls involved in organizing it. As it was a chance to meet some interesting people, who I did not realize where even in Hong Kong.

Barcamp Hong Kong, is perhaps a little different from what you might get elsewhere. First off it's not so much a techy thing. (Evidenced by the sight of 1 Linux desktop = mine...) I gave one talk on 'D', and went to a couple of others with lesser technical emphasis - drupal / OO JS etc. but the majority of talks where either business development style (raising funds, startups). or iPhone related (with out the coding stuff).

I guess this comes from the general attitude here that most programmers tend to regard it as a stepping stone to management or something, and have very little passion about the whole industry.. (as someone described it to me).

The technical talks gathered at most 20 people (out of I guess 200-300 people at the camp). And unfortunately, my talk/chat/discussion got rather hijacked by someone who wandered in, was completely non-technical, and started asking irrelivant questions. (self reminder, politely say, this is for programmers only next time)

Overall it was not much of a learning experience, rather the chance to actually meet developers with similar interests, (and see that they actually exist) - Shame I had to leave early for family stuff, as the drinks would have been good...

nested trees in mysql - handy stored procedures.

Posted: July 3rd, 2009, 12:52am MDT by Alan Knowles

Storing tree data in Mysql databases, is relatively common, however, all the existing documentation about doing this, makes the whole process rather complex.

If you google it, you will probably find the quite definitive answer at mysql.com, describing the classic parent_id method, and the left/right numbering process. Both of these methods involve rather complex SQL to fetch and update the tree.

In seeking a better solutions for a tree that was infrequently updated, but frequently queried, I thought I'd try seeing if I could write a few stored procedures to simplify the process.

Our basic database structure looks like this:

CREATE TABLE _TREE_ (
    id int(11) NOT NULL auto_increment,
    parent_id   int(11)  NOT NULL DEFAULT 0,
    seqid int(11)  NOT NULL DEFAULT 0,
    depth int(11)  NOT NULL DEFAULT 0,
    leaf int(1)  NOT NULL DEFAULT 0,
    name varchar(128) default '',
    fullpath TEXT default '',
    PRIMARY KEY  (`id`),
    INDEX qlookup( parent_id , seqid , depth)
);

our key components are

id (the nodes id)
parent_id (the nodes parent - pretty clasic)
name (the textual name of the node)
seqid - the generated order item for the whole tree
depth - how deep the node is (usefull for indenting)
leaf - is it a leaf node (eg. has no children) - usefull for icons

The trick of the stored procedures is to correctly generate the seqid, this consists of two parts, the top level wrapper and the recursive prodedure to update the table.


DROP PROCEDURE IF EXISTS _TREE__resequence;

DELIMITER $$
CREATE PROCEDURE _TREE__resequence(i_sep VARCHAR(4))   DETERMINISTIC
  BEGIN
        DECLARE v_p, v_d, v_s INT(11);
        DECLARE v_fp TEXT;
        SET v_fp = '';
        SET v_p =0;
        SET v_d =0;
        SET v_s =0;
        SET max_sp_recursion_depth=255;
        CALL _TREE__resequence_sub(v_p, v_d, v_fp, i_sep, v_s);


    END $$
DELIMITER ;





DROP PROCEDURE IF EXISTS _TREE__resequence_sub;

DELIMITER $$
CREATE PROCEDURE _TREE__resequence_sub(
    i_parent INT(11),
    i_depth INT(11),
    i_fullpath TEXT,
    i_sep VARCHAR(4),
    INOUT i_seqid INT(11)
    )   DETERMINISTIC
    BEGIN
        DECLARE v_nid, v_ex_seqid  INT(11);
        DECLARE v_name VARCHAR(128);
        DECLARE v_leaf INT(1);
        DECLARE v_fullpath TEXT;
        DECLARE done INT DEFAULT 0;

        DECLARE qry CURSOR FOR SELECT id, seqid, name  FROM _TREE_ 
              WHERE parent_id = i_parent ORDER BY seqid;
        DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
        OPEN qry;

        REPEAT
            FETCH qry INTO v_nid, v_ex_seqid, v_name;

            IF NOT done THEN
                IF v_ex_seqid != i_seqid THEN 
                    UPDATE _TREE_ SET seqid = i_seqid, depth=i_depth WHERE id=v_nid;
                END IF;
                IF i_depth > 0 THEN 
                    SET v_fullpath = CONCAT(i_fullpath, i_sep, v_name);
                ELSE 
                    SET v_fullpath = v_name;
                END IF;
                SET v_leaf =0;
                SELECT COUNT(id) INTO v_leaf FROM _TREE_ where parent_id = v_nid;

                UPDATE _TREE_ SET 
                    fullpath = v_fullpath,
                    leaf = IF (v_leaf > 0, 0 , 1)
                    WHERE id=v_nid;

                SET i_seqid = i_seqid +1;
                #// do the children..
                CALL _TREE__resequence_sub(v_nid, i_depth+1, v_fullpath, i_sep, i_seqid);
            END IF;
        UNTIL done END REPEAT;

        CLOSE qry;

    END $$
DELIMITER ;

This does the hard work of iterating through the tree, and updating the sequence number, depth, leaf field and filling in the fullpath field

Now our simple add node code just adds the node in the correct place, bumps the seqid along, so that you can then regenerate the tree.


DROP FUNCTION IF EXISTS _TREE__add_node;

DELIMITER $$
CREATE FUNCTION _TREE__add_node(
    i_parent INT(11),
    i_after INT(11),
    i_name VARCHAR(128)
    ) RETURNS INT(11) DETERMINISTIC
    BEGIN

        DECLARE v_depth INT(11);
        DECLARE v_seqid INT(11);
        DECLARE v_ret INT(11);
        DECLARE v_tmp INT(11);

        SET v_depth = 0;
        SET v_seqid = 0;
        SET

Truncated by Planet PHP, read more at the original (another 3410 bytes)

My RSS » Items by Alan Knowles

Feeds

Items by Alan Knowles

Posted: May 25th, 2010, 10:03am MDT by Alan Knowles

Posted: May 13th, 2010, 9:16pm MDT by Alan Knowles

Posted: May 13th, 2010, 8:52pm MDT by Alan Knowles

Posted: April 8th, 2010, 1:55am MDT by Alan Knowles

Posted: March 7th, 2010, 10:52pm MST by Alan Knowles

Posted: March 3rd, 2010, 1:58am MST by Alan Knowles

Posted: March 2nd, 2010, 2:25am MST by Alan Knowles

Posted: February 25th, 2010, 8:18am MST by Alan Knowles

Posted: February 16th, 2010, 8:10pm MST by Alan Knowles

Posted: January 19th, 2010, 2:36am MST by Alan Knowles

Posted: November 14th, 2009, 12:30am MST by Alan Knowles

Posted: October 29th, 2009, 5:46pm MDT by Alan Knowles

Posted: October 13th, 2009, 5:01pm MDT by Alan Knowles

Posted: September 21st, 2009, 12:16am MDT by Alan Knowles

Posted: September 13th, 2009, 4:39pm MDT by Alan Knowles

Posted: July 3rd, 2009, 12:52am MDT by Alan Knowles