Naming a Project is So Annoying

draft

If at first you walk down the dark path, forever is your destiny affected.  What you name a project may indeed be changable later, or it may not be.  So, choose wisely.

A piece of software to assemble various elements to create a single render stream for an image shall be called ... what? 

Contenders (to be updated)

    Opaque
    Wedge
    

Things I hate About Libre Office

draft

Libreoffice is the open source word processing system financed by Sun.  Its pretty good but it has several flaws that are worthy of note.

First, its help functions are completely insane, and its menus completely impossible to find anything in.  Next, even if you have saved your work, when it comes up again, it whines and complains and makes you go through a recovery process that is completely unnecessary.

Both of these two issues have the same cause.

The people who designed this software are morons.

The Insanity of Software in 2016

draft

Whether or not I was having a fabulous career today, I would still have to work hard, like everyone else, to keep up to date and learn new skills.

Since it is a part of my self-image to be very knowledgable about the nuts and bolts of doing things with computers, then that certainly means being up to date on how we implement things on the Internet, and that means learning how to write Javascript for various browsers (and in some language, possibly Javascript, for the server side).

Some of this technology is good, some of it is lousy, some of it is very good, it is all over the map. But the real problem is that it is total chaos, total insanity, and there is no rhyme or reason.

I am going to let someone else make this argument for me, and you are directed to a wonderful little article called How it Feels to Learn Javascript in 2016.

Learning Unity 3D

Unity (3D) is one of those all-in-one, inclusive, cradle-to-grave 3D packages that has a billion and one things that it tries to do in an effort to provide some sort of structure or environment for real time 3D animation, what many would call “games”. Or “VR”.

It is clear that they have done a valiant job to both provide a framework, and also to provide scriptability and programmability at every level. Thank goodness the bad old days when somehow people thought they were going to do computer animation without being able to program computers seems to be dead. It is also clear that Unity has put a lot of serious effort into providing real documentation in contrast to so many other firms out there who seem to think that they can "group source" their documentation from random users.  I appreciated that they recognized that there is user documentation online that is not very good.  They have an entertaining IDE that seems to work right out of the box and supports not one, not two, but three different but related scripting languages.  This is all good news.

The bad news is simply that with software of this type there are barriers to entry and one needs to reach a certain critical mass before you can do much of anything useful with it.  This is just a fact of life.  And at least in Unity's case everything makes sense, at least so far.  That is more than I can say about many other software packages out there whose name I will not mention, other than Photoshop and Gimp, those two I will mention by name.

So it takes time to get traction as with any serious software package.

There is one silliness which I have noticed they try to conscientiously document.  One of their three embedded scripting languages is "Javascript-like" but when you look closer you realize it is not Javascript much at all.  So that is a little weird and it has the side effect that in fact you only think you know how to program one of their scripting languages, the reality is different.

So why do I mention all this?  Its because a friend asked for some help on a demonstration he was doing in Unity and he had a little less than a week.  The problem is, in about a week one can start thinking about getting something simple done in a system like this.  A month would be more realistic.

And this is not an isolated situation. The fact is that there are dozens if not hundreds of these packages out there, each in their own niche, and none of them are terribly difficult to learn, at least up to a point. But it is not instantaneous and figuring out which ones to learn and become good at is not intuitively obvious, most of the time.

[10/16/2015 As an addendum, although it has taken longer than I like to learn elements of Unity, it is proceeding and it will be entertaining and useful to pursue this at least as far as doing a non-trivial trial application, maybe something in the so-called VR world which is certainly trendy right now]

My "Hello, world" script in C# for Unity.  From tiny acorns might oaks grow.

Potential Collapse of Civilizaton Seen as a Result of Weak Type Checking

In a world filled with the threat of war, with midterm elections that once again demonstrate the self-destructive credulity of the American people, with the collapse of the American economy due to the greed and stupidity of the American elites, is now the proper time to talk about the looming crisis of weak type checking in our programming languages?

Yes, now is the time. The need has never been greater to stop this foolish slide into moral decay and socialized health care.

The promise of weakly typed or untyped languages such as Javascript is that you can quickly and flexibly create new data structures, complex data structures, and not get bogged down by being forced to go through your entire system and make everything work together in a very pedantic, and literal way.  You can throw together arrays and lists and pass them as parameters and feel a certain pleasant lack of mental overhead in doing so.  

This can be very productive but it can also generate a false sense of correctness when in fact one has introduced a subtle incompatibility which the system is blandly ignoring for a while, only to fail in a non-trivial way when you are least expecting it.

In fact, it is shocking how well a system like Javascript is willing to accommodate undefined variables, functions, incompatible types and just keep moving along as if nothing was wrong.

But having seduced the programmer into a false sense of security, it then waits for the program to reach a certain size, or grow to more than a single programmer, and suddenly the author or authors of a program have to start tracking down a bug that comes from one side not enforcing or maintaining a data structure in the way to which it was intended, or partially implemented, or perhaps implemented but then changed due to incomplete knowledge.

The larger the system, the more people who contribute to a system, and the longer the software is in use and being evolved, the more likely this is to happen. And when it happens, one is left without the tools to find the problem other than reading the code carefully and providing ones' own type checking.

How could this lead to the end of civilization?   It can do so in two different ways.  The first is that by permitting this mental weakness, this accommodation to those who would advocate weak type safety, we are letting those who are more lazy enter into a position of responsibility in society. This is certain to lead inevitably to sloppy programming resulting in falling buildings, collapsed bridges, exploding nuclear power plants and God only knows what else.

But second, this nation is under relentless attack by inscrutable oriental criminal elements that are sponsored by their evil, slave owning, government.   Can you imagine their glee whenever they penetrate another freedom-loving web page or program in America that has been left defenseless by a weakly type-checked programming language?

We must stand firm against these efforts to leave America defenseless against these threats and rebuild American strength through strongly typed languages.

Thank you.

Using the GPU for Real Work: A Postmortem

After doing about a dozen projects with CUDA/GPU for my own edification, I made the mistake of trying to help out some friends on their project.

After working through various issues / problems I am came up with a list of somewhat obvious conclusions. I knew some of these going in, but some of them were a surprise and some were confirmed as being really true, not just sort of true.

I showed this to a friend who has spent a great deal of his career designing graphics hardware and he confirmed these and added a few of his own. I showed this list to another friend who has used the GPU commercially and he tells me I am all wrong. He always got 50-100 times speedup without any problems and things just work.

So you are on your own, kids.

Believe these or not as you please.

1. An algorithm that has been optimized for a conventional computer will be so completely unsuitable for the GPU that you should not even try to port it. One is much better off abandoning what you did before and rethink the problem for the GPU.

2. A major part of any GPU solution is getting the data to and from the GPU. Depending on what else you are doing, this could have a serious impact on the performance of the application and its design.

3. In general you should not expect to just tack a GPU program/shader/whatever onto an already existing program. You should expect to have to do major work to rearchitect your program to use the GPU.

4. Do not expect to be able to do a lot of magic things with the display and still be able to do intensive work on the GPU. Under those circumstances, plan to have a second GPU for your compute work.  I am still not completely clear on how NVIDIA shares one GPU with two very different tasks (the computers window system and your program, for example), but it does, up to a point.

4. As part of planning to use the GPU in your application, you should budget/allocate time for the core developer to work with your GPU programmer to hash out ideas, issues, problems. If your core developer does not have the time or the interest, do not try to use the GPU.

5. Debugging GPU programs is much harder than debugging normal programs. Think microcode but a little better than that.

6. Performance for the GPU is something of a black art. Small differences in algorithm can have impressive differences in the received performance. It can be remarkably difficult to predict in advance what kind of performance you are to see ultimately on your algorithm and project, even after optimization.

7. Not all GPUs are created equal even if they are software compatible.

8. And of the unequal GPUs, GPUs for laptops are particularly unequal.

9. Although the technology of GPUs and their programming is maturing, and NVIDIA has done a very good job, things are not perfect and when you run into a problem you may spend weeks and weeks getting yourself out. Examples upon request.

10. When you add a GPU to a mix of a larger application, you have complicated testing, deployment and support. If you do not have the budget for this, do not try to use the GPU.

In conclusion, GPUs are not a magic solution that just makes things faster. Under the right circumstances, performance of GPU can be impressive, but lots of things have to go right and nothing is free.

Unless you are my friend who says that GPUs just work and speed things up. In that case, I guess they are free.

Unspeakable Evil, Esoteric Knowledge and WebGL

What madness possessed me that day I will never be able to explain. Having accomplished what needed to be done, why did I press on?  I do not know myself what unconscious self-destructive character flaw caused me to innocently go forward and attempt to learn the forbidden knowledge that lurks just below the surface of WebGL.

Those who watch, those who control, those who oppress our pathetic lives through means both grand and small,  yes even through those apparently innocent graphics API committees that design our APIs. 

Or put another way, I decided to move forward and add texture mapping and multiple objects to my little WebGL object viewer which is really just an excuse for learning WebGL.  

1. I attempted to use the feature, attribute arrays, to download the UV coordinates for use by the shader. But even though I have used this feature successfully for the vertices, colors and normals I could not make it work for UVs.

2. Texture mapping would not work. After a while I figure out that my texture map is not an exact power of two, and that seems to be fatal. First, I do not think that is clear in the documentation. Second, why is the only error code that WebGL seems to know about is "error"? Why can there not be more informative error messages?

3. I use a trick to get the UV coordinates to the shader, and that works, but I can not use that trick in the final software.

Yeah !  A cube with a texture map !

4. In the process of writing slightly more sophisticated shaders, I discover that (a) functions are tricky and that (b) it is not clear which GLSL (shader language) WebGL implements unless you read very carefully (the answer is that it implements GLSL ES 1.0 only) and (c) there is no obvious documentation.  There is documentation it turns out, but you have to go searching around for the documentation for OpenGL ES 2.0's 1.0 shader language. Confused?  You should be, there are many, many OpenGL shader languages defined out there and they are all different yet similar. 

5. In the process of doing this, using textures and so forth, Firefox on Windows gets a hard failure because I am not using vertex attribute 0.   Really, why does it care?

6. Once I start using texture maps, Google Chrome, which had been very compatible before, now fails hard with the following security error.

Security violation?  What security violation?  I am loading a texture map, ok?

Here are the conclusions I drew from this not very fun 72 hours: 

1. WebGL is fragile. You may think you have learned how to do something in WebGL, but its possible that you did not and that you got lucky. You may not be able to use that feature in the future, or you might, but you do not know.

2. WebGL is designed to a very low level. Even the API for the 2D canvas is at a much higher level (and much more usable) than the 3D API.   The only solution seems to be to write your own management system to do basic stuff that could or should have been done for you.  

3. The WebGL error system is nearly useless. It does not give enough information to diagnose problems and errors.

4. The WebGL documentation is problematic. There is no reference manual, there is no shader language documentation (it turns out there is, but you have to be a detective to figure out which one it is and where it is), there is way too much reliance on group sourced internet tutorials that are very low level and often incorrect or unhelpful.

5. It is not clear how much a problem browser compatibility is. Initially there did not seem to be any problem but as of yesterday I am dead in the water and I can not figure out what it is complaining about. Therefore be warned, browser compatibility is a real issue.

6. This might all just be a problem of software (im)-maturity.  Maybe it will all get better with time.  Maybe someone will write a Javascript package to smooth over the loose ends and everyone will use that API instead of the WebGL one.

7. One does what one always does in this situation.  You write your own software to hide the ugliness of the underlying system and after a while the problems go away.   The irony here is that OpenGL was originally created to make it easier for people to start doing 3D.   It has evolved into something that seems to take pride in making it as difficult as possible for a beginner to do 3D.  Weird.

There are a lot of positive things to say about WebGL (including the ability to run a 3D application from a browser which means that the end-user does not have to install a program). But if I were advising a client on whether or not to commit to doing an application that was important to the company using WebGL, I would tell them "Maybe, but proceed with caution". This is a very low-level API, it is not yet robust or documented. One will need to do tests and carefully evaluate whether what you need to do can in fact be done in this environment and at what cost.  Look before you leap.

More on the Design of WEBGL and the Origins of .OBJ

We have made a little progress in the last few days on both WebGL and the .OBJ origins issues.

This is an image from the current .OBJ object display software written for WebGL and written in Javascript/DOM/HTML.  Now, I doubt anyone actually needs a .OBJ viewer, its just a way to get familiar with displaying geometry with WebGL.

Instead of just showing you a picture, as I do, below, I should be able to just give you a URL so you can run it yourself, but I can't because I don't actually have a place to post the files for you to address (without seeing advertisements or in other ways compromising or inconveniencing my friends). The only web server I have access to is my own and it has only temporary IP addresses which Facebook will not even let me post. Besides, why bother to post a temporary IP address? I think it is funny how money controls everything, without money, our society says, don't even try to do anything.

So for fun I am posting this dilemma here and on Facebook, maybe someone will know where I can post some number of html and js files in a way that can be accessed by people on the net without inconveniencing them (e.g. no spyware, no ads, no Doubleclick tracking) at a price that even the poor can afford.

Regarding WebGL, once you realize that the designers had gutted it of any even slighlty higher level 3D and that this is a very low level interface to some of the power of the GPU without in any way providing even trivial conveniences for the programmer, everything is fine and you proceed.

I think they did this for good reasons. In the past, by making computers easier to use we opened the doors to allowing people to come in and take away our livelihood. The kind of conveniences I am talking about are trivial and do not hold back an experienced 3D technologist in the least (after they waste their time trying to figure out what the people who redesigned OpenGL were thinking ...).  (1)   So all that happens is that they inconvenience those who are writing their first 3D software and also tend to make such programs written to be a little more obscure because they will all be different (having to reinvent the wheel is trivial here, but since everyone will do it differently naturally it will be a little harder for someone to come in and read and modify because they will have to learn a new convention rather than rely on an industry-wide convention.)

I have no problem with this and in fact applaud it. As a victim of globalization and of the spread of these technologies, anything that makes knowledge that I have more arcane and valuable can only be good.

Regarding the .OBJ matter, we have learned both more and less. The ideas and the general approach to what the .OBJ format became was in the air at SIGGRAPH and the U of Utah in 1979 according to Frank Crow. According to Julian Gomez, there was no software from E&S that would have been an example of a .OBJ like format that would have come from them.   My leading hypothesis had been that the people at Abel's had received some sample software to (for example) load a geometric object and display it on the Picture System.   What would be a normal software development approach would be to start with that program and then write their own software on top of it.   Its an extension of the old joke that Unix only had one device driver (in other words, that everyone modified the original device driver when they had a new device to interface).  Julian may have written one of the first such "display an object" program at Andrews AFB long ago, but it did not use anything similar to a .OBJ-like format.

Julian also suggests that Mickey Mantle, who was in charge of customer support of E&S, may know more.

Thus it is still quite possible that the .OBJ format as we know it was based on ideas that were generally available but not based on specific code from a sample program, as I had expected. In other words, it may still turn out that the Abel version, written initially for motion control preview for Star Trek: The Motion Picture is the one that escaped into the wild (as I believe) and that it originated at Abel's, even if based on ideas more generally available.

 The time when this would have been written by the way is about 1977.

There is more to say, but also more to find out. I have been unable to find any early E&S documentation, for example, which would answer a lot of questions.

If anyone knows how to reach Mickey Mantle, please let me know.

_________________________________________

1. Well, that is not totally true.  OpenGL used to support drawing polygons, convex polygons, directly.  Now only triangles are supported so you have to subdivide into triangles yourself.  The problem is, this is a problem whose general case is actually fairly difficult.  Why not make everyone re-solve this themselves? Its good for them, it makes them think about fundamental principles.   I have no idea what these people were thinking.