Categories
article graphics shaders

New Toys

I have been interested in computer graphics for a long time but never really interested enough to make some positive steps toward it. Like a lot of people, I have tried to get a basic app working in OpenGL or DirectX but never really got very far. It was all a bit intimidating.

However things have changed. These days, on DirectX, there are a load of fantastic tutorials on the internet, as well as seriously helpful libraries. If you’re interested in getting some physics working there’s a few good books and often the content is backed up with source code. For WebGL (which is a flavour of OpenGL) there’s great tutorials to help, and really cool insights into how it works.

There’s quite a lot to take in before you can really grok what the graphics pipeline does, whatever API flavour you chose. It was whilst trying to figure out how shaders work that I stumbled across some stunning examples of what is possible. What I didn’t realise, at first, is that the examples I was looking at are simply fragment shaders.

If you don’t know how 3D graphics works you might say ‘so what?’. But let’s just say it’s not the easy route to getting 3D computer graphics done – not to me anyway. The mathematics involved looks in a lot of ways harder, but the programming looks way easier. Programming this way is mostly declarative, there’s no bonkers API and there are far fewer loops because the magic is in the power of the GPU and the shader loop.

My only problem is that shadertoy.com doesn’t quite work how I want it to. For one thing it keeps timing out (guess they need more funds, so I added some through Patreon), but that aside I wanted a bit more control over the shader and how it can be embedded. Partially for this blog but also to learn a bit more WebGL.

That happened two weeks ago. I’ve been messing around trying to get something working for this site and making some embeddable shaders that I can embed directly here. I think I’m almost there …

Categories
article

Server timed out. Rebooting.

TL;DR: The hat is back!

It has been over 10 years since I last wrote something on this blog. Some usual life stuff happened that I won’t trouble you with. The rain fell, but mostly, the sun shone.

After 5 years of letting the site rot I’ve just spent the best part of 2 days getting it back up, it’s had:

  • OS upgrade;
  • WordPress upgrade;
  • but most importlanty a new logo!

5 years of rot has gifted me about 4,000 accounts created by SEO spam-bots so in a fit of rage I deleted all registered accounts. As far as I can tell there is very little spam (if any) on the site, which brings me to my next point.

I’ve disabled comments. I realise, a blog without comments isn’t really a blog at all. But the problem is, as the number of spam posts increase it becomes painful to moderate and so I don’t. Turning off comments is fairer to potential commenters, who might write a comment and never have it published. At least I’m saving you some time.

I’m looking into alternatives, but so far in all the usual places. Perhaps I’ll find something that addresses my problems with spam that doesn’t involve Google. We’ll see.

After all the work of getting the site up I felt like I should probably write something. I’ve been doing some interesting (to me) hobby research into knowledge engineering and separately, computer graphics and shaders. Alongside my day job of project management, there’s a lot of potential topics, and perhaps some of those musings will end up right here – in the hat.

To keep the pace up, the promise I’m making myself is that new posts will be shorter. So … let’s end it there and see what happens next.

Categories
article programming

The Spread-able System

Spreadsheets are everywhere. They are simple to create and are an immensely powerful tool. Unsurprisingly then this means that a lot of areas of business rely on spreadsheets to function correctly. But spreadsheets are dangerous too. They suffer from, well-known, fundamental flaws.

The problem is that spreadsheets are a special type of code, and I’m not talking about the Excel ‘macros’ I’m talking about the formulas. As such they probably need to be treated the same way as other types of code, but their very nature makes this difficult. But I’m getting ahead of myself, let’s first look at some of what is good and bad about spreadsheets.

Pros

Spreadsheets are remarkable for their:

  • Utility – we can bend them into almost any shape we want because they give one way to represent almost any business process;
  • Portability – we can pick up our little gobbets of data and logic and relocate them to almost anywhere inside or outside the company, in file-systems, mail servers and web-sites;
  • Simplicity – you don’t have to explain a spreadsheet to anyone. They might have to be a proto-genius to figure out how it works but the working knowledge they would need to get started is pre-loaded in their heads and ready-to-run.

Cons

So they sound pretty useful, and I like to think that I’m a pragmatic guy, so why do I hate them so much? Many have noted about the shortcomings of spreadsheets. The page on spreadsheets at Wikipedia spells it out clearly enough so I’ll paraphrase:

  1. Productivity – Working with spreadsheets requires a lot of “sheet-shuffling” to reach the required goal. The bigger the sheet, the more time is spent copying, cutting and pasting cells around.
  2. Reliability – Although what consitutes an error in a spreadsheet is subjective, the paper A Critical Review of the Literature on Spreadsheet Errors” (pdf) reveals a series of studies (some more recent than others) that have shown that approximately 5% of cells contain errors.
  3. Collaboration – Sharing a spreadsheet is difficult. Having two independent people working on the same sheet and merging their results is as far as I know impossible.

The first two items don’t bother me overly. Yes, it’s a problem but then the alternatives aren’t that great either. Consider what you would do if you didn’t have a spreadsheet to fulfill the task. You’d either do it with a bit of paper and a calculator (i.e. simulate a spreadsheet) or get a programmer to do the task for you. Either way the amount of productivity loss/gain and the amount of errors aren’t going to be that significantly different from using a spreadsheet. Don’t get me wrong, I love my fellow programmer, but we make a LOT of mistakes too. The difference perhaps is that bespoke systems usually end up getting audited (and hence fixed) and spreadsheets often don’t. Although this point is probably moot.

Good + Bad = Too Bad

My real beef is with what happens when you have the ‘pro’ of high portability with the ‘con’ of low collaborative power. You have no way of knowing which version of the spreadsheet you have is the “true” one, and which version is duff. Every copy, whether it be inadvertently through forwarding a sheet by email to someone else or explicitly by taking a ‘backup’ is a 12 foot tall baby-eating, business-crushing monster waiting to rip you and everyone you love apart.

Hug the Monster, Then Run

The thing is we kind of have to embrace the baby-business-beating monster because it’s about all we’ve got. There are some tasks, as a programmer, that I’m really happy that you as the non-programmer don’t bother me with and solve yourself in sheets. Want to set-up an intra-company phone-book as a spreadsheet so you don’t have to bother will all that “Access” voodoo? Be my guest, but I’m watching you. Want to set-up a spreadsheet to run your fantasy football so you don’t have to add two numbers together? Go right ahead, I’ll even drive you to the game so you don’t miss the turn. Want to set up a spreadsheet to calculate payments and and do a mail-merge with the results … STOP. RIGHT. NOW.

The truth is though that you might not know that you’re creating the mother-of-all spreadsheets when you start. I might not know it either but there will probably come a time when a line is crossed and then I will want to know what you’ve been doing and who you’ve been doing it with. I’m just like that.

Unless you are small company (and hence don’t have a lot of choice) you have to be very afraid of trusting anything that might lose you money to a spreadsheet. You need to be very aware of the risks and the potential-costs you are letting yourself in for. Here in Europe there is even a special interest group dedicated to highlighting the risks of spreadsheets. Those guys must throw wild parties …

The Missing Links

In my opinion there is something missing, something that can fill the gap between spreadsheet and system.

I think we need something that can:

  1. Track spreadsheet changes – Not knowing which spreadsheet is “true” and which lies (by being able to identify revisions of the sheet that have happened after yours was ‘branched’), and not being able to merge sheets is a problem. Perhaps someone solved it already, if they had that would be great.
  2. Track spreadsheets themselves – Having some more information about what sort of corporate-data was being accessed, who was using it and how frequently they ran it might alert us to potential spreadsheet monsters being born.
  3. Narrow the gap – Making spreadsheets more like traditional software systems, without significantly castrating the usefulness of the spreadsheet, would be great too. This is a little like asking for the moon on a stick though.

Perhaps I’ll make something like this one day. I have to admit it’s not a terribly exciting project but it has some potential I think. Perhaps I could spice it up by throwing a party and invite the guys from the “European Spreadsheet Risks Interest Group”. Now we’re talking. How will I budget for the 7-up, party hats and streamers? In a spreadsheet of course.

Categories
article

Educate A Business Person Today!

One recurring theme I have noticed with users of systems I’ve worked on, is that they aren’t nearly as stupid as I think. They often try to make sense of the system thrust in-front of them. This is mostly out of necessity since it stands between them and them doing their job, and so they need to make sense of it. Having written some truly awful systems myself I wish all of them the very best of luck.

Another, seemingly unrelated observation, is that business people are truly astounded, and often suspicious, of how long it takes to provide a solution to a particular problem. Writing software is simply hard so that partly explains it. Sometimes however some of the solutions that are asked for can come quickly. This might happen if the system was expressly designed to handle new cases of the particular solution being requested or if producing the solution requires little more than a configuration or script change. Or it might just be dumb luck that the release cycle has worked in their favour.

Disconnect

This disconnect between implementation times, with no apparent reason to the business user can cause problems. Sometimes it feeds the suspicion that they are being ‘had’ in some elaborate con:

“If change ‘x’ takes a week then surely change ‘y’ should take half as long. How could it not? It only takes half as many words to say out loud. Those guys in IT need firing.”

When that business person is a manager it can lead to awkward situations for developers:

“Change ‘x’ took a week, change ‘y’ will take half as long. How can it not? It’s the only thing that stands between us and product success. If it doesn’t I’ll fire those IT guys”.

The simple truth is that unless a business person has also the developer’s view of the system they will not be able to make sound judgements about it. Hell, I have a developers view and not even my judgements are particularly sound.

However humans are pretty adaptable creatures, and rather than telling them the answer we should explain the answer in a way that they can understand. If they want to listen then educating them has a few potential benefits. For one thing it might make you look like you care about your users, rather than being that IT jerk who steals everyone’s food from the refrigerator. However, if you get your point across without sounding (to them) like a lunatic then you might improve their mental model of how the system actually works.

Breed

There is a breed of programmer out their in the world today that has either evolved or engineered themselves into a situation where they are the only one who ‘knows’. Yes, you know who you are. Sometimes they do this as a survival instinct to make themselves indispensable, sometimes because they’re not great communicators or educators. These are the people that need to be fired because their value is way-less than they think it is. They actually harm the productivity of the company by being obstructive or uncommunicative, plus they’re a real pain-in-the-ass to work with.

Yes you will need to keep a watchful eye on your newly educated fledglings. Especially the managers, but there’s nothing knew about that.

Categories
article programming

Floating Point Flotsam

I have never been particularly clear about when to choose single or double floating point arithmetic. I think I have been operating on a sort of ‘trial-and-error’ approach for some time. It seems ludicrous that I should not, after all these years, know exactly when to chose single or double precision arithmetic. The truth is that I don’t, and it’s now time to fix that.

First, a little background. I was, for reasons that are too sad to explain, interested in how to calculate (accurately) the time of the Vernal Equinox. That is the exact time of the year (to the minute – if possible) that the sun is 90o above the earth at the equator. Anyway, I found some code which is an implementation of an astronomical algorithim designed expressly for the purpose of calculating the vernal equinox and is accurate to about 20 minutes. Which is good enough for now.

Now the next piece of background is that I was also doing this in Lisp and up until now I have let the reader interpret all the literals I enter. When I tried to do the calculation in the REPL I could get no closer than being in the same part of the day as I was expecting and with the result somewhat rounded to the nearest half day. After some head scratching it finally occurred to me that the the computation required more significant digits than I really had. It seems that the formula I was entering was being interpreted as single precision floating point numbers because if I had wanted double’s I would have suffixed my literal numbers with a ‘d’. It would seem that d could also stand for D’uh. Seems fair enough. Time to do some research then …

You see, according to the IEEE standard 754-1985 a single precision floating point number has 23 significant bits and a double has 53 significant bits. For me to answer the question of how many significant figures I can get in decimal would require me to know that the smallest decimal fraction that I can represent in binary is. This number would be 1/223 which is about .00000011920928955078125.

Now, floating point numbers are represented from a fractional part and an exponent part to give the decimal representation of the number. Therefore you can never get more accuracy than the smallest binary fraction multiplied by the exponent you have. This means that the significant figures should be something like:

log10(1/(2^23)) = -6.9

Therefore when a number has 7 significant figures you are already losing a little accuracy, the more significant figures you add the worse it will get. It was then clear that my astronomical antics were less than stellar since the first literal in the computation has 11 significant figures.

Indeed whilst I was thinking about this problem it occurred to me that if I wanted to continue using single precision I could split the fractional part from the integer part and continue this way. However, this is still inferior to a double because it would give me 7 significant figures for both parts and therefore a total of 14 significant figures. This is inferior because using my shiny new brain I can show that a double will give about 16 signficant figures. Of course I could have also concluded that double’s are better than two floats by noting that 23 bits+ 23 bits < 53 bits but that would never have been as much fun.

Type Word Size Mantissa Dec Sig. Figs.
Single 32 23 7
Double 64 53 16
Extended 96 63 19
Quad-Extended 128 113 34

You could argue that I could have saved myself 20 minutes of time by looking the answer up but then again I would never remember the answer unless I proved it to myself first. So, now my spring occurs at the same time as everyone else’s and I know why. Oh it happened already? Sheeeeiiiitttt.