Even in the days of Web 2.0 you encounter many errors when it comes to Uploads via HTTP. The protocol was never made for uploads or better to say: there are still enough bugs in implementations and the protocol itself.

Very familar, e.g. is the problem of having a meter on upload progress. And i also don’t want to mention the problems of WebDAV implementation of Windows for now, or the limitations of firewalls or proxies…

But let’s have a look on solutions provided today in the Web for Uploads…

Flash/HTTP

We have a modified version of R6 running on a webshop with regular uploads of 20-70MB files. If upload goes to fast – like on business lines with upstream ≥3Mb – Flash is likely to throw errors telling the visitor a script is running too slow. Then the customer has to choose „continue“ or to „cancel“, which usually ends up in canceling the job

This behavior is caused by Flash calling the „onFileChunkComplete“-Event (or something) too often, or let’s say after every 4096 bytes sent, which can result in many many calls if you have a good upstream bandwidth.

Second, there are also those „unknown errors“ in Flash caused at runtime. The source yet is unknown and it seems to continue with Flash10 and  some little amount of customers smaller than one percent share. Sorry, unknown means unknown and Adobe won’t give support here! we’re back to Flash9

Yet the same thing happens to JavaScript if update goes too fast and too often, e.g. by using a progress bar constantly being updated (20-50times in the second). Therefore I had to split up the graphical update process to a seperate class that just goes on update intervals calling the current/total-values of the FileUpload-Class.

Well using Flash as upload client is nice, especially with the ability to use real sockets. For Flash9 the HTTP methods don’t provide the possibility of splitting large files – so you could upload them in seperate chunks. It’s about  security.

Then Flash10 possibly could do now (by accepting local files), but here we have the issue of having the upload beeing fired from Flash itself – scripting doesn’t work anymore, so no hidden flash uploader, that could do the job

But customer complaints still continues… especially form those using MSIE and don’t get a thing right.

PHP and APC, for now. HTTP again…

Okay, usually you gotta think this works now. It’s out now for long enough.  But fail of proof!

I leeched some script form PHPRiot yesterday by Quentin Zervaas, fixed some bugs (while(1=1) is great idea in JavaScript ), added a file limit check and hoped i finally would have something that really works.  But it didn’t:

• Multiple file-fields do not work with APC
• If file uploaded is greater than far greater than limit, apc throws out 0-values for current and total
• Sometimes just seems stop working somehow; but after an apache restart apache it works again

I tried so many ‘solutions’ now and somehow it really seems to me, that monitoring the upload by calling a server-side status-script always has some problems. Probably it is really an Apache-issue… we’ll have too see…

Sliced HTTP Upload via Plugins – a solution?

I already mentioned it in the Flash/HTTP section, and indeed it’s more safe, because the file chunks are simply smaller!

I did many tests with Apache, and often it just happens that Apache closes a TCP connection on big file uploads. The source is yet unknown to me. Probably it has to do with Apache itself. I didn’t encounter the problem when using FastCGI and a Perl-Script to handle my upload.

So, i guess doing sliced upload is a good option. But then you could only use Flash10 which has other restrictions or Java. Yep, that thing that only works on 70-80%  of user browsers, MSIE for the rest that reports about having Java supported – true indeed – the other problem is: does it really run on those often misconfigured and compromised customer systems?

Well Flash10 now does, but have some limitations we probably could live with:

• load() and save() APIs can only be called in response to user interaction (such as a button click).
• locations of the loaded and save files are not exposed to ActionScript.

Since the API is now asynchronous (non-blocking),

Final match: HTTP vs. FTP-Uploads in the client

• HTTP was not made for upload!
• use more bandwidth than you actually need (base64!)
• HTTP is not resumeable (but you could retry on splitted chunks)

Let’s summariuze: You can only monitor what was uplaoded by having control over your client and you can only split files, when you have access on it.

Use a Java-Applet or code a Flash-Client that works over FTP! And use a FTP-Server that is secure and can handle virtual users, like vsftpd. Then code some backend,

Yes! Probably it is a solution. But you still have the problems with users not supperting Java, or who complain about Java „loading“ too long. Here are some other good tries, that usually work:

• http://www.aurigma.com/
• http://www.jupload.biz/
• http://www.javaatwork.com/
• http://upload.thinfile.com/

We’ll see further when the first FTP libraries for Flash 10 are finished.

You just can’t! Because you only communicate with SimpleXML by getter and setter methods. It’s possible there is another interface – anywsy i doubt it!

So, the magic of adding nodes is to convert from and to DOMDocument. The snippet below is just something stripped from sourcecode, so you get the idea how it works.

Damn ugly but that’s just the way it is!

I was a friend of Zend Framework when it was in version 0.5 because of the convincing and really well-thought concepts. Most of them are even better than those of CakePHP (ok, CakePHP can some little other frameworks can do much more than Zend Framework). But Zend Framework is now final and already 1.01, and i guess it has a bright future

There is an easy to understand and some good tutorials on Ralf Eggert’s Blog that gives you an introduction to MVC and how things work with Zend Framework. Just try it for yourself!

And for all trolls: I know Java is better!

An APC from the aliens movies; picture py paramount studios

 

Okay, i probably guess that probably have some experience with it, probably already made an implementation that works. And probably you only think it works. But often not for production level or enterprise! At the end of the article you will praise once again APC – the PHP Opcode Cache

Well, I tried it all, the PECL uploadprogress_meter-extension which doesn’t really always compile to the latest PHP, upload with Flash ≥8 and JavaScript (because we want to show the progress in HTML). Then we or i have also additional requirements to the upload, like posting variables with one file. And it should really run stable what most solutions don’t!

Let’s summarize – Progress Meter using PHP as primary server language:

PECL Extionsion uploadprogress_meter

Often only compiles and runs stable to the version it was built for

Wrappers with Perl

Megaupload, Filechucker or how they call them, have some problems in stability, resulting not always in writing the monitor file. Who knows why? I made an own version based on Megaupload (which should be on release here shortly)

Other PHP hacks

Do they really run stable with the current PHP built?! Who will update them?

Flash-Uploader with JavaScript Backends

While beeing client side they have the generic problem of often calling a JavaScript-Callback that results in a flash-side timeout alert-box when you’re using broadband connections >1MB/s upstream. Shitty for all the customers on your commercial website with such a connection and without any technical knowledge.

Pure Flash-based upload progress

A solution that really works and runs stable, but do you wanna make your whole site in Flash or are you working in Flex?

APC – the PHP opcode cache

You heard right! Just add an apc.rfc1867 = on to your PHP ini and submit also a APC_UPLOAD_PROGRESS>-field and your on the right side calling the progress like with apc_fetch('upload_'.$_POST['APC_UPLOAD_PROGRESS']); For more just have a look here and enjoy

My Perl-based solution mentioned above (called jjUpload) will be available here shortly – didn’t i already post about it? It won’t work on it anymore since APC now has that functionality. Just bad for all you ZendCore- or ZendDebugger-Whores

Anyway, i’d like to hear what you prefer – client side progress-meters -or- if the status should be pulled from the server?

Every PHP programmer should know that.

I stumble upon too many lame md5-hashes based on md5(rand()) or even more ciritcal crc32(rand()). That could really lead to trouble especially when you’re using a lame unique id for critical applications like payments or somethimg.  The chance that one id is used two times is not great, but if there is a chance for such an error, a programmer has to eleminte it!

And it’s so easy with PHP!

string uniqid ( [string $prefix [, bool $more_entropy]] )

Gets a prefixed unique identifier based on the current time in microseconds.

For more see the PHP Manual…

Schon einmal geärgert, dass man bei PEAR alle Pakete von der Hand installieren muss? Nein,man muss nicht!

Zum einen hilft es oft pear config-set preferred_state beta zu setzen, da viele Pakete sich nich im Beta-Status befinden.

Und uum einen hilft download-all, was aber alle PEAR und PECL-Pakete herunterlädt.

Eine andere Möglicht ist mit pear remote-list | awk ‘{print $1}’ > pear-packages  sich erst einmal die Liste aller Pakete zu holen, diese kann man dann bequem in einem Text-Editor bearbeiten, und per cat pear-packages | xargs -n 1 pear install -a dann auch bequem alle ausgewählten installieren