←

Well, a tumultuous relationship with Yahoo Pipes has ended after a long struggle with various bugs, inefficiencies, annoyances and finally service refusal. There’s no doubt Pipes is a very useful and clever system, but it has some very serious downsides:

• Caching – every pipe is rigorously cached, and updates are so infrequent as to allow bugs to go unspotted during testing, and suddenly rise after the fact.
• Crap editor – I know YQL allows you to actually type out your stuff, but Pipes forces you to use its extremely buggy visual editor, which (among other things) fails to resize properly, does not allow copy and paste, frequently puts operators inside the wrong loop (and thus overwrites a painstakingly written item creator or whatever), seems to suddenly stop working at random – especially if one of your feeds is broken or returning malformed data, which results in ALL of your pipe, even unconnected nodes, breaking ingloriously.
• Usage limits – despite the fierce caching, they still apply a fairly unsatisfactory usage limit, and without warning will shut down your pipes if they go over the limit.
• Inefficient operators – sometimes you really have to go several times around the block to do very simple things, and you’ll quickly run into these issues.

All in all, I know Pipes is free and efficient, but upon my pipes being blocked because of a bug at my end which made too many requests, I had to quickly write my own PHP version to do the same job – and I found I had more control, it was on the whole an easier experience, and I have control over my own caching and debugging.

The people who use pipes are invariably developers – so as a word of warning: if you can do it yourself, take the time to do so, don’t rely wholely on third party services.

Just thought I’d share a little trick I’ve just discovered while anally going through a couple of my projects making all the HTML properly indented… it seems I spend a lot of my time trying to find ways to make my life in some way easier, and this is one of the ones which worked!

ob_start();

// your code here

$pretty = new DOMDocument("1.0");
$pretty->preserveWhiteSpace = false;
$pretty->loadXML(ob_get_clean());
$pretty->formatOutput = true;
echo $pretty->saveXML();

Essentially it formats your XHTML (you’d have to use Tidy for regular HTML, but we don’t like that dirty stuff) correctly via PHP’s own XML and DOM manipulation classes. Very nifty – though of course it adds a bit of overhead, and makes everything much less forgiving as far as not well-formedness goes, but hey – I consider that an incentive to write better code 🙂