The Computer Upgrade Dilemma in June 2023

 

 This is a draft.

Update Alert:  Apparently the next generation threadripper pro is months away from announcement and/or release.  It would make sense to wait a few months and see what happens.

This is a discussion of the issues one might consider when building/upgrading a home studio that is used for 3D rendering and some machine learning.  It has been over 4 years since I upgraded my computing here in Santa Barbara and the world has changed several times in that time period.  What I thought would be a straightforward review of existing technologies scaled to my budget has turned into a several month slog through all kinds of things I wish I did not have to think about.  I review some of the issues involved below.

There are two different goals.  

    1. Upgrade my main Linux workstation to improve rendering performance.  

    2. Buy a completely new, modern technology workstation which would be used for both rendering,     localized machine learning, and development.

The 1st goal was motivated by simplicity and cost.  I am using a 1st generation scalable Xeon, and bought an entry level CPU for it.  There are many more CPUs available, some of them of very modest cost (so it seemed). The second goal could take longer, and fixes a problem I had with limited PCIe bandwidth in the 1st generation of scalable Xeon, especially with the UP configuration or when you did not populate both sockets of a dual socket server.  The budget of the latter system would be more expensive, but would solve both problem sets at one time.  The budgets might be $1,000 and $5,000 respectively.

But there are many issues, some expected and some unexpected.  Issues include:

-- Is there a good difference between what is a server and what is a workstation?  The provisional answer is that a workstation has a higher base frequency, has features such as audio on the MB, and has a chipset to handle a bunch of small-change IO such as legacy USB. 

-- What are the limitations of the current configuration beyond raw processor power?  Am I memory bandwidth limited?  Cache limited?

-- What are the real costs of a better 1st or 2nd generation Xeon?  Why are there seemingly great deals from unknown vendors but the top vendors keep list prices which are seemingly high for 5 year old technology.

-- Why does Intel make so many different versions of each generation of Xeon.  Specialization varies as to the extent of the market?  Which ones would I want, or can afford?  Not all these processors seem to be equally available from the reputable vendors.  

-- What warranty do the "really good price" vendors of refurbished processors offer?  My experience with 1st generation Xeon says that you may go through several CPUs before you find one that works, but that once it works it just keeps on working.

-- The AMD threadripper pro is about to be updated.  Should I wait a few months for the update?  But the updated system will certainly take advantage of newer technology which means among other things a higher memory cost.   Do I really care between DDR4 and 5?  PCIe 4 vs 5?

-- All of these processors seem to have a "base speed" and a variable "turbo speed" which it automagically uses when it has to.  Well it will only do that if several important conditions are met, one of which is that the processor must be cool enough to allow this without exploding or melting.  The only way that could possibly work is if we are using liquid cooling for the CPU.

-- My experience in this area a few years ago said that you had to be able to return several non working components before you got one that worked.  Choose carefully.

- One issue has been whether or not I save $700 or more and reuse components from the current workstation.  Power supply, chassis and disk.  Do I really want to destroy a working system, however slow?  Hasnt that often been a mistake? 

-- As alluded to above, there are real differences between current generation AMD, next generation AMD, 4th generation scalable Xeon and their 5th generation coming out end of year.  Whatever you do youre fucked.

What fun.  All you need is cash and these issues go away.  Thats what the real people do and if you cant just spend you dont deserve anything good.

 

 

The Pernicious Impact of Frivolous use of Ancient Computers (JOSS #1)

draft

JOSS was the RAND Corporation's internally developed timesharing system to permit one computer to be used by many people (researchers) in their work and in a way that was more convivial than punching cards or paper tapes with machine code.   JOSS stands for "Johnniac Open Shop System".  In researching this important early system, I came across this beautiful description of an unexpected and, in the eyes of a key developer, an undesirable effect of making JOSS more available to researchers at RAND.

The researcher wrote:

In the summary and conclusions section of RM-5270-PR

The Need for Reciprocity When Insulting Someone With Social Media

One of the great aspects of Social Media is the ability to offend people at a deeper level and with more oomph than mere email or online forums. In the past, using receding media technology, such as writing on paper or the printed word, one needed a little time for the insult to become clear. But now with the bold new technology of Social Media and the power of the Internet we can mortally offend someone with the click of a mouse.

And we do, we do insult people, we insult people all the time with Facebook.

But since this is a totally new approach, some mechanisms may need refinement until it settles down into a truly democratic method of punching someone in the ego.

Such refinement is probably necessary in the case of the Facebook method of blocking and unblocking. As it stands now, one can block someone without their knowledge, but also unblock them without their knowledge as well. Blocking keeps either party from seeing anything about each other, even to know that the other person exists on Facebook. So far so good.

But there are issues.  The first problem is that the blocked individual has to find out on their own nickle that they have been blocked, which is always a demeaning thing to have to do. One wonders what happened to the other person, one searches, one does not find, then one discovers that one has been blocked. I have noticed that by the time this happens that the feeling is usually, but not always, mutual. In my case, the two times I have been blocked it is because I tried to build bridges to someone who I have damaged relationships with. In both cases their blocking me is a rebuff.

But you see, having been rebuffed in my effort to open communications, that is pretty much the end of the matter from my point of view. But Facebook does not give me the ability to implement that. The other party can unblock me and see what I am doing whenever they want, and I don't want that. I want any unblocking to be mutual, they have to ask, and I have to agree.

Because, frankly, I really don't want to see or hear from them again.

Google Mail Uses Esoteric Knowledge to Create Your Avatar

Things move ever swiftly on the Internet, that bold new paradigm, in order to maintain valuation and return maximum value to the shareholders. In the never ending efforts to achieve these worthwhile goals and with all the advanced and esoteric technology being used, it is reasonable to expect that mere users may be confused by what they see and foolishly and incorrectly believe that they are witnessing bugs and mistakes. It is incumbent on all of us to keep up and be aware of these misperceptions and help the future-challenged user to understand that the system is without flaw. 

For example, in the process of receiving and processing email with Google Mail, I was unaware that it helpfully creates a visual avatar of all the people who send me mail and does so flawlessly and automatically without needing to review its results with the mere sender of the email. It is not permitted for the user to be aware of the clever, neural network based, deeper learning informed, higher-level consciousness algorithms that Gmail uses to implement this miraculous new feature because we are uninitiated in the Mysteries.  Still some of us have speculated that Gmail might look out on social media, or perhaps remember attachments that the sender has used in the past.

I suspect it does nothing so simple. I think it is probable that Google is using the esoteric knowledge associated with the ancient Hermetian Mysteries themselves. Many have tried to use these Mysteries for practical purposes in the modern world, but I believe that only Google has had the resources and intellect to actually accomplish this difficult task.

Are Consecrated Masters using Esoteric Knowledge to create your Gmail avatar?

Alas, my sister in law did not realize the perfection of these esoteric algorithms and so when I made a comment about the nice picture of the dog that Google had used for her avatar, she tried to rip my head off.

I think it is important to remind the users that they are indeed "mere users" and that they should not attempt to understand the ways of the Illuminated Masters.  They should accept that their life is an open book, not just to the FBI but to the world at large and anything they use or say on the Internet may be repurposed for their own good.  Google Mail may choose all or none of these ephemera to represent you to your friends, to your family, to your clients, in fact, to anyone. You have no control over it, nor should you.

You should have faith that the Illuminated Masters working in secret shrines will use these Mysteries to create your new image and that the result will be as perfect as the spheres with which the celestial bodies orbit our planet.

Where did this come from?  No one knows but my sister-in-law was not amused.

Foolish mortal! Give up that illusion of control, abandon yourself to your fate and accept that the Adepts of the Mysteries have your best interests at heart.  

Centos Linux 6.6 Good, 7.0 Bad

When a new major release of an operating system comes out, I generally wait six months or a year, and then try installing it on a non-critical, non-production machine that I keep for just this purpose. That way I can keep doing serious work on my main systems while I work out the issues with the latest release.

In the past this has worked out well for me.

I have been very pleased with Linux as a workstation environment as well as a production server environment. Although not real Unix, it has certainly been very reliable and surprisingly scalable, which is remarkable I think.

For those of you who do not know your Linux distributions, Centos is basically the same as the major Redhat releases, but without the Redhat logo and without the formal support (for which of course you are expected to pay).

It has lots of people working on it both at Redhat and out in the world and although it has its quirks, as all Linux distributions do, I have had really very little to complain about beyond the usual issues that one faces when there are too many solutions to a problem and it is not clear which one to use.

For the first time, I have tried a new release of Centos and backed off to the previous release. Several things happened to cause me to do this.

The major annoyances all came down to the situation that the desktop part of Linux was not getting the attention that those of us who use Linux as a development environment would desire. In particular, the X window system does not automatically come up any more, and you have to jump through hoops to try and make it automatically start. Also, a variety of features that I have been pleased with in Gnome (one of the two desktop environments) suddenly went missing. Third, there were serious performance issues after a period of use that I could not explain, and did not seem to have anything to do with the usual things one looks at in tuning a system. It made the system both cranky and unreliable.

But the final disaster that just was unacceptable, is that this release of Linux insists on installing system patches and updates whether or not you want it to, destroying the old version, and imposing the new version on you. The problem is that it did so, but did not deliver a working version of Linux. The system had been automatically trashed and I then had to decide how I wanted to recover when there was no easy fix. Some of this brought on because of brain damage of the Linux community involving graphics drivers. I use the Nvidia driver exclusively and that may have complicated things.

But it is an ironclad rule here at my place of work, that updates are not installed until I want them to be installed just to avoid this kind of problem. This is not an isolated incident. I have in the past had very bad experiences where kernel updates were made and the OS stopped being usable. But in this case it is not obvious how to turn off the automatic updates and I dont want to fuck with it.

So for my uses Centos 6.6 is infinitely preferable to Centos 7. It comes with a working window system, a working desktop, is more reliable, and doesnt self destruct whenever it wishes to.

I dont know what this means for the future, and that does worry me.

Reading List on Data Storage for the Computer Illiterate

This is the second post in the boring “build a backup for your studio” series of posts. The first post is here.

The primary reason I am writing these boring posts is the occasion of having a friend of mine, a professional photographer, recover from a catastrophic data failure.  Whenever I would bring up terms like "network file server", she would put on that expression of "I am just a girl and I dont know what that means" that so many of us are so familiar with.   The good news is that even my brilliant professional photographer friend can pick up these computer terms with very little effort.

This stuff is not hard to understand.  What is hard to understand may be how things are implemented to work well, if indeed they do work well, but the basic concepts are straightforward.

The fact is that most professional users of computers, even those in their own home office or studio, will have a heterogeneous collection of files that look like they are all attached to the local computer even though they are not.  Some OS's handle this better than others "out of the box" but they all accommodate it.

Most of the time you, the user, do not care if a file is local, or on your local network, or even further afield. But you very well might care if you are your own systems administrator or your studio architect and since most of us are our own administrator, you have to know this stuff.

So get over your computer anxiety and gender bias and get this done.   Here is your Wikipedia (and one optional Dell white paper) reading list.

1. All your files on a computer is managed by a file system.

https://en.wikipedia.org/wiki/Computer_data_storage

https://en.wikipedia.org/wiki/File_system

2. Most simple storage on your basic home computer is directly attached storage.

https://en.wikipedia.org/wiki/Direct-attached_storage

3. All modern computers today also support network attached storage.

https://en.wikipedia.org/wiki/File_server

https://en.wikipedia.org/wiki/Network-attached_storage

4. Whether your storage is direct or on your local network, there are a variety of techniques designed to take these relatively cheap disks for personal use and make things more reliable. There are a variety of ways of doing this. The simplest is disk mirroring. RAID is a way of formalizing some of the existing techniques of combining multiple disks into a more reliable, or better performing, “virtual” disk.   You mostly only care about RAID 0, RAID 1 and RAID 5.

https://en.wikipedia.org/wiki/Disk_mirroring

https://en.wikipedia.org/wiki/RAID

5. RAID can be implemented in hardware or software or both. People used to care passionately about which one they had, hardware or software. The reality is that you should not care which one it is as long as it is reliable, fast and low maintenance.  For those who think they care, here is a Dell white paper on the topic (optional).
ftp://ftp.dell.com/app/3q03-Dum.pdf

6. But a file system, or a file server, or a reliable disk subsystem is not the same as having a backup system, although it may be a part of that system.

https://en.wikipedia.org/wiki/Backup

Now we can get on with the exciting yet boring design of our backup system.

The Boring Topic of Designing a Backup System For Your Studio

[This post was prompted by the occasion of helping a friend try to recover the data from her failed disk server. So the annoying details of this problem, and the necessity of dealing with these issues, is on my mind.]

As part of a series on designing, building and running a small computer animation studio, we are going to have to discuss backups. I will try and break it into small pieces because, frankly, it is a real bore. When we started using computers we did not do so for the joy of making backups which is like taking out the garbage, its not our first choice of how to spend our time. Furthermore, it turns out that there are choices to be made here, and real design issues. I am sorry about that. It just is.

When people started using computers, probably no one told them that they were now expected to be responsible adults about how they cared for their data or run the risk of losing it.   But all of us who have been using computers for a while know this only too well.  You can learn from our mistakes and save yourself a lot of trouble.  

When you drive a car, you are expected to learn how to drive safely. When you work at a real corporation or a University, then it is likely that your professional work is already being carefully backed up and protected, at least to some extent.   But the rest of us, at small companies or on our own, have to put our own system in place.

Keep in mind that hard drives, big or small, solid state or otherwise, are not intended to be perfect.  They have a known failure rate, and even though the manufacturer knows that some of their disks will fail, they only know this on the level of probability.  Disks are made in batches and the failure rate of disks within a batch are estimated as that is part of creating a warranty for the drives.   But disk failure is not the only cause of data loss.  

So here are some basic definitions and principles. In later posts we will go over some of the design choices you may have to make, are likely to have to make, when you design your studio.

For those of you who think I am less creative because I worry about such things, please go fuck yourself. Thank you.

1. The place where you do your professional work might be called your office, or it might be called a studio. A studio can be for one person or 1,000 people. The work might be your personal artwork, or your personal financial records or it might be a very expensive collaborative technology and creative project with a $100M budget.

2. All of these offices and studios need to have given some thought to how much protection they need to give their data in case of disaster, what is the likelihood of disaster, how much it is worth to them to lose one days work, one month's work, one year's work, etc.

3. The goal of a so-called backup system is to provide a level of protection for your data if disaster strikes for any reason, whether by computer malfunction, act of God, or human error.

4. No backup system is perfect, but different backup systems provide different levels of security at different costs, where costs means varying amounts of capital, costs going forward, attention that must be paid to maintaining the system, technical expertise and so forth.

5. A simple backup system well executed is better than a technically complex system that is over the head or beyond the needs of the intended user. An expensive or technically complex backup system that is not well implemented or maintained may be worse than no backup system at all.

6. A backup system is holistic. Together it provides a level of protection.  If some of the pieces work and some do not, you may still have a level of protection.  Thats the plan.   But it is better if all the pieces work generally speaking of course.

7. Backup systems are usually layered, that is, you have more than one protection so that if one fails you do not lose all data, but can fall back to another level. Generally this is implemented as a system to improve the reliability of the main file servers combined with discrete backups saved in a vault from earlier periods.

8. Backup systems are probabilistic. There is a probability of disaster, a probability that any one backup will not be readable. No backup system is perfect, but a good backup system will make the probablility of losing all your data much less likely.

9. Backup systems must be tested before they are used or you run the risk of not finding out that there was a problem until it is too late. This is an extremely common occurrence.

10. No one but you can judge whether this effort, these costs, and so forth are worthwhile. Only you know what this data is worth.

and finally,

11. I have found over the years that I never had too many backups.

In a later post we will go over some fundamental design choices and the kind of risks you will need to protect against.

Should We Abandon the "Rational Actor Model of Filmmaking"?

Is there too much bad computer animation in today's movies?  Is that even possible?

I continue to see people out in the world, on Internet forums and blogs, complaining piteously about the alleged overuse of bad computer animation in film. Here is a recent example pointed out to me by the people at www.io9.com.

Six Reasons Modern Movie CGI Looks Surprisingly Crappy

http://www.cracked.com/blog/6-reasons-expensive-films-end-up-with-crappy-special-effects/

Is it possible that there is too much CGI, particularly bad CGI, in modern filmmaking?

No, of course not. Everything done with computer graphics in visual effects is exactly as it should be and the audience should agree if they know what is good for them. But sadly, some among the audience, a pathetic few, have not gotten the message. Two messages in fact.

The first message that these whiners have missed is that the modern art of filmmaking is all about the bad use of computer graphics: that is its very raison d'etre. That is its highest goal, second only to maximizing shareholder value, of course. When the audience sees computer generated garbage, that so-called garbage is nothing less than the manifestation of the new art which demands new artists and perhaps new audiences as well. Some of these filmmakers, like Michael Bey, may be far ahead of their time. But it is the duty of the real artist to lead and society will follow along eventually.

The second thing to realize about the tsunami of shit that we see in computer-generated visual effects is that it is not merely a lack of skill on the part of the effects providers, although that is often true as well.

Those who kvetch must look further into the heart of the madness itself and realize that it is almost certainly the filmmaker's vision that is up on the screen. If it is ugly, it is the ugliness that the client wanted. Bad computer animation has been incorporated into the filmmakers body of work and sensibility: it is an element of their style made manifest.  Admittedly, sometimes unconvincing or sub-par work is the result of a lack of skill on the part of the VFX supervisor or facility, but even then it may be that this apparent lack of skill is why these specialists in the computer arts were chosen. Their aesthetic matched that of the filmmaker's and a perfect harmony was found in stupid visual effects. It is not accident that things look the way they do.

To paraphrase a gem of wisdom from our friends in Communist China, “The fish stinks from the head”. In other words, when something smells bad to understand why it smells bad, you must look at who is running things because what you are seeing (or smelling) is probably what they asked for or represents who they are in some manner.

Yes, there are details in this vision that we can be critical of. It does seem that many do not realize that a camera must act like a real camera or it will cause the failure of the suspension of disbelief. The failure to embody the characters with appropriate gravity or weight is often cited, although that is but one example of the bad animation which we are regularly exposed to. The failure to realize that visual effects is about sleight-of-hand, it is about making the audience see what you want them to see and not about number of pixels or “photorealism”. The failure to realize that too much of anything is counterproductive.

But in our new Globalized and virtual Hollywood, nothing succeeds like excess. There is something about visual effects done with computers that can cause a producer and/or director to lose all sense of proportion and just throw 3D computer generated shots at their movie in lieu of thinking.  Perhaps this is a way to compensate for their own sexual inadequacy?  Perhaps the filmmakers have developed an anxiety disorder associated with working with a writer?  In the future, will 3D animation be classified as some sort of dangerous drug that causes the victims to peck without restraint at the lever that releases a 3D CGI pellet to the drug-crazed pigeon-filmmakers?

Should we now abandon the "rational actor" model of filmmaking, which says that those who are making this expensive entertainment product are reasonable and talented human beings doing what they think is best for the kind of entertainment they are trying to make?  Have our artists been driven mad by the opportunities which 3D animation have revealed?

Or is it something else.  Could it be that our overly-critical audience swine, who the Germans refer to as negativenpublikumschweine,  must look within themselves to find the real problem?   Perhaps it is not "bad" computer animation per se that they are reacting to, but their own provincial point of view that is not sophisticated enough to understand the director's vision?

The Anomaly of Enjoying Jupiter Ascendant

George Lucas has famously said that movies are binary: they either work for you or they don't. If they do, then you ignore any little flaw. But if they do not, then every flaw or potential flaw is noticed and used against it. I do not know if Mr. Lucas was original in this observation, I doubt it, but ever since I read that I have noticed that he seems to be correct. All movies have flaws of course, but when you are caught up in a film, one is happy to ignore the problems and issues that in other circumstances would be seen as deadly.

The question then becomes, what leads a member of the audience to lean one way or the other? From acceptance and enjoyment to rejection and boredom or worse? One aspect of this choice may be what is called the “cockroach in the salad” effect. Lets say you are eating out at a fancy restaurant and you have ordered a salad and when it arrives the first thing you see is a disgusting cockroach on top wiggling its antennae at you. You call the waiter over and he removes the offending cockroach and salad but the damage has already been done. It will be hard to get beyond that terrible first impression. On the other hand, let us say you are at a restaurant and see nothing that you particularly want, but you order something and to your amazement, it is really good. From that point on, everything works for you.

So my argument here is that the basis of cultivating a positive impression of a creative work is a mashup of “first impressions” with “low expectations”. If you did not expect much, then getting something really good is likely to push you over the edge to a positive impression. And vice versa. Until you get that push, whether positive or negative, then you are in a state of uncertainty. Is this film any good or not?  

I don't have any other way of explaining the apparent anomaly of enjoying “Jupiter Ascendant” (JA) a film I was born to hate. What could have caused this odd reversal of expectations such that I actually enjoyed watching this film? Can Science explain this or must it always remain a mystery?

Consider the following:

First, we have a dinner sequence in which our plain jane heroine introduces us to her Russian extended family in America. Its actually very funny. You mean the Wachowski brothers actually have a sense of humor? How would we have known?

 Jupiter's sister before and after a special bath.  Ah, refreshing !

Second, we have a classic theme in fantasy fiction, the “person of noble birth who does not realize that she is of royal blood and possibly the heir to the throne”. In this sub-genre, the kids are separated from the adults by the manner in which it is revealed that our average neighborhood girl is actually “her majesty”. In JA this is actually done quite well and unexpectedly. A fight sequence between two alpha males upsets a hive or three of bees which scares the shit out of our female lead, but no need to worry, the bees have been genetically programmed to treat “royals” differently and so our two fighting alpha males break off their sparring to recognize that something quite odd has happened. The babe has been revealed as a member of a royal family of some sort.

Bees show the way

Third, after our plain jane babe has had her butt saved by our hero, she tries to encourage him to ignore her royal birth and kiss her. He refuses, revealing that he is not really a man, but closer to a dog, or a wolf. She comes back with a splendidly stupid response: she has always gotten along well with dogs. Its does not persuade. This is funny.   It occurs to me that in certain ways this incident is itself a flaw in the movie. If our hero was really descended from a dog, then he would have no trouble taking advantage of the situation.  But I digress.

Fourth, the sequence with the candles and certain aspects of the wedding sequence are lush and clearly represent the director's desire to have a big wedding one day.

Fifth, the two brothers of this royal family are monumentally fucked up.   Parents, take note, do not name your child "Titus", it never seems to work out.

What a creepy asshole this guy is.

But most of all, who could not like a movie where the lead babe keeps having to remind people that "I am not your damn mother"?

So what we have here is an overdone, weird movie in the same genre of, for example, the original Dune novel: a space opera with exotic economies, insane royal familes and fight scenes between things bred to be good at fighting.

Overblown, a misfire, there is no doubt that it is a miscalculation on a galactic scale, truly a stupid movie.

Nevertheless, as a 12 year old, emotionally and psychologically, I found it often to be an entertaining movie and was willing to overlook its tragic flaws.   I also have a tendency to hunt and peck at my movies.  I am not bound, like so many are, to watch a film from beginning to end.  I prefer to "sample", sometimes with the sound off, in order to better appreciate its higher values.   When properly used, this technique can improve most movies.

Postscript

It did not hurt that the uber-schmuck, John Gaeta, was somehow deposed from his role as visual effects supervisor on this semi-epic. I shudder to think what manner of ego-swine must have replaced him/it. I am sure that Mr. Gaeta will enjoy a new career in the food service industry or some other profession worthy of his talents.

Yet Again, the Problem is the Documentation

There are several unwritten rules about the Internet and we might as well make them clear up front. The first is that everything is great, and if you dont say and acknowledge that its great then you are an asshole and must be ignored and written off as someone who complains.  And I do complain so they are correct.  The second rule is that the documentation sortof sucks, and it does.  It is not intentional on anyone's part that the documentation sucks, or rather is uneven.  It is just the way things turned out.

Now for some details and a specific example and one more time it is not the technology per se that is bad, although of course there are always things one might like to change.  The problem, as always it seems, is that the documentation is either wrong, inadequate or overwhelmed by noise that masquerades as signal.  And that noise manifests itself as "helpful" documentation available on the Internet and authored  by the "group mind" that is, unfortunately, wrong or out of date or replicates what is already there or all of the above and there is no easy way to tell the difference.  As a result, the anarchic state of the documentation makes learning new and possibly better approaches on the Internet annoying and much more time consuming than it needs to be.

Websockets is the “new” approach to client server communication for browser applications. It does not look like much, but it is apparently almost as good as what we had with the Arpanet on day one in 1972.   As I read more about Websockets, I realize that there is a lot of thought that has gone into it in fact just because the Internet is not the ARPAnet and there are a variety of considerations that this forces on the design of technology like Websockets.  

Now Websockets is marked as experimental and is also considered to be incompatible between various implementations/browsers. However, it seems that is old news and that there are good implementations in most browsers and a variety of frameworks to hide differences between browsers.   For my application, I am not too concerned about this as my specific application is more of a proof of concept and we can finesse such things as working transparently on all browsers, for example.

But as always, the documentation is ad hoc.  There are many different frameworks one might use for your server side implementation.  Each of them has a different approach to documentation. Just choosing between the different frameworks (in this case that works with node.js) is itself a chore and a half.

For example, the websockets.org site has the source to an echo client that runs in a browser and is written in Javascript, and they also run a live echo server on their site.   But the source for their echo server is not available.  Why not?  And there is no contact information on their website such that you could ask them that question or any questions at all.

I presume that the people involved in all these technologies and frameworks are not lazy nor stupid.  I suspect that there is a combination of things going on here.  They include such things as (a) being not particularly talented at writing documentation nor enjoying the process, (b) not realizing that such documentation is necessary, (c) balancing the needs of this project with other responsibilities, (d) relying on someone else to do it, and (e) actually believing the groupsource myth that says that other people will write it for you.

My guess, my personal guess, without enough information, is that Websockets is an effort by an elite who simply do not understand or care that people learning their protocol who have not lived with it as they have on their committees will need more documentation and usable examples to make good use of their time.  It works for them.

If you dont like it, well its the Internet, and you dont have a choice.

[Addendum.  As time goes by, I penetrate more of the mysteries and it is not too bad. In fact, it may even be reasonable.  But Jesus, they really don't try to make it easy for you.]

Internet Advertisement Shocks Viewers and Advertisers Worldwide

In a shocking advertisement placed before a trivial news story for the British Daily Mirror viewers were subjected to an unexpected video.  Although claiming to be an ad, the short film presented in a calm voice without any hype information about how to use a popular spreadsheet program in a new and useful way. There were no flashy graphics and no helpful links trying to entice the viewer to go to another web site. It was simply a short informative film with a helpful technique.

Advertisers throughout the world were outraged by this violation of all standards of public decency. “They may have well as slapped every advertiser in the face with a dead fish”, said Bill Plotnick of the International Brotherhood of Advertising and Commerce, “Advertisers everywhere were stunned by this gross violation of protocol”. “Advertisements are meant to be intrusive and obnoxious”, he went on to explain, “that is how they attain their value. If we don't have potential customers cursing us with foul language we know that we have not done our job”.

Testing has revealed that viewers actually watched the controversial video all the way through instead of clicking on the “Skip Ad” button further raising hackles on Madison Avenue.

“This sort of thing has to be stopped at the source, “ said a spokesman for Turid, Turid, Turid and Snap, an Internet advertising agency located in Zurich and New York. “Governments must enact legislation to prevent this from happening again and the perpetrators of this stunt fined or arrested.”

When asked if the Department of Justice would file a complaint in British courts to stop the rogue advertisement, officials there declined to say more than they were investigating the situation.

Daily Mirror Homepage

http://www.mirror.co.uk/

Secret Parameters in Firefox

Please be sure not to tell anyone that the secret parameters in Mozilla Firefox are listed under "about:config".

The Plausibility of Getting Medication in Mexico

In order to clear up a misunderstanding... the interest in getting medication in Mexico is not predicated on a desire to save money nor it is to avoid getting proper prescriptions.  It is because DEA and Ca. restrictions on certain medications make it impossible to get these medications under various circumstances (e.g. the pharmacy will not sell them to you) even with legitimate prescriptions.   

One function of the Internet which can be said to be socially neutral is to share information among people who have a common interest, whether that interest is sailing, jihadist terrorism or getting legitimate medical help in a region of the world. This feature is certainly of positive value some of the time, but it is also of (culturally dependent) negative value other times. Clearly we do not see advice on blowing people up as positive, but those who seek to right what they see as wrongs do.

In that spirit I am going to discuss what I think I know about getting the medications I need to treat ADHD in Mexico. One of the few advantages of living in Hell is that I am near the border of Mexico. Its an hour away.

This note is not intended to aid people figure out how to get recreational drugs or anything of the sort. It is intended to be a personal journal of my attempts to get legitimately prescribed medications for a disorder that for reasons known to the DEA are difficult to get in this country.

I also happen to believe that American adult citizens of good standing should be permitted to be able to choose what they take in their bodies without the consent of doctors or government as long as they are of proper age, and do not do stupid things like drive motor vehicles under the influence. Which of course they do every day of the week under the influence of the most dangerous drug of them all: alcohol. But that opinion of mine is not relevant to this and related posts. This post is solely about getting properly prescribed medications that are not disbursed in this country for a variety of regulatory stupidities.

There is a lot of misinformation about getting meds in Mexico on the Internet and I am probably adding to it. As time goes by I will add links to the useful sources I have found. So far, I have not tried any of this and I have not been to Mexico for years. This information comes entirely from reading dozens or hundreds of posts on the Internet and forming my own judgement. No doubt I will try some of what I discuss here if for no other reason to satisfy my curiosity. Repeat, none of the following is personal experience.  And I despise giving what appears as advice without direct personal experience, but here we are.  Any numbers quoted are numbers found on the Internet and are likely to be “internet numbers” which is to say not true, or only somewhat true.

First notes, and then tentative conclusions.

Americans going to Mexico to get medication is a vast business. There are numbers like 40 million visitors per year just for this purpose. There are numbers such as $200M a year in pharmaceuticals bought by Americans and carried back across the border. Remember, these are internet numbers, taken with a grain of salt.

There are at least a thousand pharmacies in Tijuana that exist to service this business. These pharmacies fall into two classes. Only the much more rare first class handles what we would call controlled substances. The more common second class of pharmacies handles the more common and uncontrolled substances.

The vast majority of purchases fall into a few classes: older people seeking to save money on the medications they need to survive or not be in pain and who are looking to ease the appalling costs added to medications in American in order to extract more money from innocent victims. As you might imagine our government is outraged that sick and poor people would try to get the same deal that large corporations get by going offshore, and do anything to stop it. But for some reason the border guards will permit these prescription, but non-controlled or scheduled substances back across the border if they are carried by the person they are for and are for personal use. I know very little else about what is involved in getting these medications and they are of no interest to me, at least not yet. The other major group of purchasers are young men looking for Viagra at a discount or other appalling date rape drugs which are apparently easily available. There is also a big business in self medicated antibiotics and people stock up on these. This is the sort of behavior that makes doctors in this country and their paid servants, the politicians, mad with rage and they do whatever they can to stop it. There is another category of people who are seeking medications not yet available in this country but are available in Europe. Mexico seems to follow the European approvals.

Many of these medications are made directly in Mexico in factories run by the major pharmaceutical companies. There is a lot of discussion about what is and what is not of adequate quality. My feeling is that this should not be a major concern unless you are doing things like cancer drugs or heart disease drugs. In my case it is not a concern. The medication will work in the manner that I expect or I will not repeat the experience. If it doesnt work I am no worse off than I was before.

Generally speaking your American prescriptions are not good over there. Mexico has their own system of prescribing medications and if you are interested in so-called controlled substances you will need a Mexican doctor to write you one. This is apocryphally not too difficult it is said. See notes below. A photocopy of an American prescription or empty prescription bottles *may* be of use in demonstrating that you are of good will and it *may* be useful in talking to a licensed Mexican doctor in getting a prescription for what you need. On a personal note I plan to take with me photocopies of a relevant prescription and an empty bottle or two just in case they turn out to be useful.

It is apparently common for a person to go to a pharmacy, be directed to a doctor, and get a prescription for what they need for what we would consider a nominal sum, e.g. $20 or $30 US. Again take this all with a grain of salt.

The doctor may wish to break the prescription down to small amounts and suggest you go to separate pharmacies.

Coming back across the border seems to be the following. You are supposed to declare anything you buy of this type. If you do not, and they find them, you are guilty of a misdemeanor and what you bought are likely to be confiscated. If you do declare and they are for personal use, generally they let you through. If you are carrying so much that they think that you are going to resell them, then they will confiscate. It is not so clear to me that any of this applies to controlled substances, but if they are small amounts for personal use, and you have copies of legitimate prescriptions or American bottles, then it is believed they let you through. I consider this a major flaw in the whole scheme because it is very likely that the behavior may depend on who is manning the station and what the enforcement flavor of the month is.

You will need a passport.

Tentative conclusions:

For non-controlled substances that are not health critical, such as heart disease, this is likely to be a way to save substantial money and has no apparent risks.

Those who use medications that are health critical need to be more careful and you can find discussions on the internet of how to do so.

Carrying an American prescription and or empty bottles may be helpful in getting the medication or coming back across the border, or it may be totally a waste of time.

You will need a Mexican prescription for anything that is controlled in Mexico. This includes most but not all of the controlled substances in this country. For ADHD, anything useful is controlled in both countries.

Coming back across the border is a dubious activity that may be helped by the medication being for your personal use, and having proof of legitimate American and Mexican prescriptions. It is critical that this medication be for your own use and not for resale. Failing to declare these things is a crime, do not do it, it will get you into trouble.

You will need an American passport.

Plan to spend all day on this activity, at least the first time you try it.

In conclusion, I am leery of the whole process. It is an ambiguous venture, not quite tantamount to drug smuggling but perhaps one of those odd holes in the control of trade materials across the border. But we are told we live in a world of globalization and that it is ethical to destroy American livelihoods by using slave labor in China, a vicious dictatorship, so why shouldn't Americans try to get a little savings by going to Mexico.

The hypocrisy of our system is rampant, overt, and starting to get annoying.

Do Not Leave Those Naked Pictures of Yourself on Github

One of the not-so-entertaining results of our civilization's obsession with computing and cash is the "fuck our customers we don't care" approach taken by the consumer electronics and computing industry towards such things as systems administration, backup administration and security administration and its impact on our lives.  Now we are all forced to take on these dreary sub-specialties or face one of many horrible fates that this technology mania has brought down upon us: the wiped disk, the non-working backup, the zombie computer used by Chinese or Russian spies, or worse, the "hactivist" holier-than-thou swine ready to exploit your assets to mine Bitcoin or some other juvenile and anti-social goal for their self-appointed crusade.   We are all now responsible for these and many other tasks and woe unto those of you who think you are above such things for then your sins as documented by your iPhone will appear on social media and there you will be, in full color, engaging in some drunken bisexual orgy as an undergraduate for everyone to see just as you are running for your first political office or other responsible position.

Be warned, if you wish to avoid this or some other horrible fate, there are a few hundred things you need to pay attention to at any one time, although that list is a moving target. You have to know enough to keep yourself out of trouble.  No one else will do this for you. 

Many of us use Github as a repository for source code for our projects and collaborations. In the past I have used it off and on, but these days I use it more or less 7/24. As part of your repository, one could keep security strings that give access to various other resources that exist out there, such as the Amazon cloud. A friend did just that and forgot about it. Although he certainly knew better a few years later he made that repository public (it was either that or delete it, he wasn't working on that particular idea anymore).

Well his repository contained security information for his cloud account on Amazon which he also wasn't actively using and some hackers grabbed it and ran up a bill in the many 10s of thousands of dollars per day. Amazon.com caught it nearly immediately and my friend will not be liable for most of this bill, hopefully not any of it.

My friend is beating himself up because of course he knew better. He does know better, by the way. Don't let this happen to you. He suggests reading the following discussion on these issues to learn how to keep passwords out of your Git repository.

http://programmers.stackexchange.com/questions/163506/how-does-one-handle-sensitive-data-when-using-github-and-heroku

Never forget it's a jungle out there and that, generally speaking, people are scum.

Pacific Grim

Warning: This essay contains spoilers for the film Pacific Rim (2013).

Some are born great and some have the mantle of greatness put upon them.

In our world there is mere craftsmanship, then art, then great art, then movies with giant robots.

But not everyone who is called to the altar of greatness is up to the challenge or in some way evil or circumstance interferes with its realization, and we experience the tragedy of a movie that could have been important but that fails and lives down to our worst expectations.

Such is the case with Pacific Rim (2013).  I had seen the effects reel at the Academy Bake Off and I was looking forward to the director's take on the critically important sub-genre of fiction: giant robots beating the shit out of alien and hostile ocean monsters.  But the great concept for the movie was let down by a truly puerile script and shallow characters. The humans were so stereotyped and uninteresting that even the giant alien monsters seemed more richly drawn, realistic and authentic in comparison.

Lucas claims that a movie is binary, that it either works for the audience and they ignore the flaws, or it does not work for them and the flaws are completely annoying.   I think that this principle of all or nothing has merit.   For example, I did not notice many of the flaws in Edge of Tomorrow (2014) when watching the film and when they eventually did occur to me it did not really bother me.

The flaws were made less important because the film was so entertaining.

A mashup between a German scientist and a Cambridge University mathematician, or something.

But not so for Pacific Rim, at least not for me.  There were so many problems and all of them attached to a very obvious and banal plot.   Not even giant robots could save this movie from its plot.   Here is a short list of just some of the problems in no particular order of importance:  1. If you are dying of radiation poisoning, you don't just get a nose bleed, or rather if you do, its because you are bleeding at all your orifices.   But none of that really matters because the radiation has probably killed all your blood stem cells and you will be dead in less than a week, horribly, and wont have the time to lead a group of desperate men and women in a last chance struggle to save humanity.   2. I found the cultural stereotypes of the German/Cambridge scientist to be offensive, although it was supposed to be funny, 3. The mind melt with the alien thing, aside from being improbable, is just confusing. Do the bad aliens read the stupid little scientist mind or not?  4. Umbilical cords are generally for mammals, as I understand it. Are the filmmakers saying that these hideous underwater alien monsters are descended from mammals? That doesn't seem very likely from what we know of them.  5. This may sound silly, but what is the motivation of our alien menace and the big fellows on the other side of the breach?   I mean what is going on?   Are they just attacking because they think its fun? Are they after our women?  What?  6. These big aliens although they are impressive looking seem rather average in terms of construction.   If they can be filleted with (for example) giant spinning sushi knives or a sock to the jaw then it seems logical that they would respond well to a couple of dozen standard, stand-off, air-to-air missiles, not to mention MK48 torpedoes.   7. Its all very well to throw around words like "analog" in regards to EMP, but for that to work that would mean that all the control systems of the adorable Gypsy Danger would have to be analog computers, etc, and I kind of doubt it.

Open wide and stick out your tongue... 

Ok, enough.

It is a principle of visual effects that great visual effects will not save a bad movie.   That is certainly true in this case, but there are some things to note about the film that are positive, in terms of cost reduction, costume design, production design and, of course, visual effects.   

1. They probably saved a lot of money on the writer.

Many people feel that having a script in a visual effects movie is just throwing good money after bad. Certainly, Michael Bay has never been held back by not having a writer on his films.  They probably saved several hundred thousand dollars on this one item, which would leave them more money for visual effects. 

2. The female lead was given an excellent costume.

Movies of this type are often calculated to appeal to adolescent boys of all ages, and one way to get their attention is to put your female lead, suitably cast, into a skintight and/or polyethylene outfit. Actress Rinko Kikuchi plays the role of Mako Mori, the spunky and strong female technocrat and martial arts specialist. I think that the rubber/latex outfit that they have her wear while controlling the giant robot in partnership with our hero is very practical and shows off her intelligence among other attributes very well.   I am still looking for the right single frame to show you what I mean, this image is a standin for now.

The properly sexist still of Mako in her latex jumpsuit has eluded me so far.  

3. In visual effects, objects interacting with water is very difficult to achieve in a realistic manner.

4. One reason that water generally looks fake in earlier visual effects (see WW2 movies or pirate movies with ships generated with model photography on a pond or swimming pool) is how off the sense of scale is, no matter what the visual effects people did.   We get an excellent sensation of scale for most of the important fight scenes of the film which either take place in shallow water or under water.

5. Finally, it is a non-trivial thing to give these 3D models a sense of scale while they are beating the shit out of each other.   And most of the shots, although often ridiculous, were also dramatic and did have good scale to them.   I was very impressed.  

ILM did all these things very well.   Have a look at some of these stills and remember that IMHO the only way to really judge the work is in motion and on a big screen.

6. Very few movies get to show what it is like on the other side: to show the unspeakable and unknowable alien world. This movie did that acceptably I thought, the pacing was good and the reveal of the horror that is the vast and evil alien intelligence is suspensful... Of course the implacable menace is just beginning to understand that they are doomed as disaster overwhelms them.

Pleasantly abstract, the alien menace sees their doom approach.

Its the stories and the characters and the details that let them down. The movie feels like it was written for 10 year olds, and maybe it was. Only a 10 year old could go with the hackneyed characterizations and the stupid plot points. And of course the whole premise is ridiculous. Although it would be moderately expensive (1) and messy to turn these monsters into shredded fish food, it would be straightforward to do so with the weapons at hand in any modern air force or navy.  I mean they are big and ugly and spray acid and look pretty mean, but it seems to me that they blowup pretty much like normal flesh and blood, alien though they may be.

The other scientist stupidly visits the alien fishbait abortion.  Nice eyes.

But I prefer to emphasize the positive about this movie and hope that something better will happen next time. Good art direction and creature animation does not a good monster movie make. del Toro has the capability of doing great work, I hope we will see better and more moving, plausible, end of the world implacable monster movies from him in the future.

________________________________________________

Notes:

1. The cost of a Hellfire missile is roughly $70K but I think that is a little underpowered for this activity.  A Tomahawk missile from Raytheon is about $600K - $1M depending on how you look at the accounting. A Mark 48 torpedo is roughly a million a pop, I think.   Delivering these munitions is not cheap either.  We are talking about flying F-18 Super Hornets off of aircraft carriers or of attack submarines delivering many torpedoes as suitably modified for this application.   So it would be completely plausible for the destruction of one giant alien sea monster to cost at least $50M and probably more like $100M  if not more per critter.   Not cheap, but immensely cheaper than what is portrayed in the movie as the last hope of mankind.  No nuclear weapons would be necessary.

Tomahawk Missile

http://en.wikipedia.org/wiki/Tomahawk_(missile)

Mark 48 Torpedo

http://en.wikipedia.org/wiki/Mark_48_torpedo

Pacific Rim on IMDB

http://www.imdb.com/title/tt1663662/

In Response to an Anonymous Comment on my Review of Android

I wrote a scathing review of Android many months ago, intentionally being sarcastic and demeaning to what is clearly a very successful operating system. I am resisting looking up the numbers of users of Android (not necessarily programmers, end users) because I am sure they will be stunning, however much they may have been inflated by marketing scum. The reality is that whatever the numbers are, this is a very successful operating system by any measure. But all the more reason therefore to have expectations regarding the quality and the way potential developers are treated.

You can read this review here: Toaster Oven or Computer? How Can We Tell the Difference?

The point of the current post is to reply to a comment, anonymous, that I just received that suggested, tersely, that I should revisit my review of Android. The implication being either that it has changed, which I doubt, or that I am stupid or that I am wrong.

But before I begin, let me respond to a potential criticism, that I simply want new technology to be the same as old technology which I already know. No, I don't, but thank you for insulting my intelligence and motivation.

I doubt very much if Android has changed. The kind of things I was responding to are fundamental to the system and could only be changed if the people at Google wanted to change them. But it is the nature of such things that they are not changed, at least not willingly. How many psychiatrists does it take to change a lightbulb? Only one but the lightbulb must sincerely want to change. Either they do not give a fuck, or they think it is great, or they dont know any better, or it was a design goal to make Android completely incompatible with everything else and have not considered what is involved for a non-Google person to do development on the device.  It may be that they are only interested in the participation of organizations with a lot of resources, and the lone developer has only marginal value.  (1)

Second, I do not think I was being unfair. I think that I got a good sense of what the design philosophy behind Android was and what Google thinks of their users (which is that they are going to do things their way and spend the time and energy and money to get up to speed, no matter what that costs). Remember this project has essentially infinite money compared to most projects in this world. If they wanted to do a good job, they could, but to do so would require that they want to do a good job and that they hire someone with the aesthetic and design sense who was capable of doing so.

It is my opinion that computers are first and foremost about aesthetics. There is very little about computing that is not about aesthetics in one way or another. Yes, there are other criteria as well which are more mundane, some obvious, some not obvious. But generally speaking, the values of those who have written or designed (if there was a design) the hardware or software have been implicitly expressed in their design and implementation.

There is nothing new in the above paragraph. This insight has been common knowledge in computing since at least the 1970s and probably before and in the other arts for much longer, centuries at least.  It is one of the reasons, perhaps the primary one, that Macintosh consistantly has a good user interface and Microsoft does not.  The author of a work necessarily reveals who they are and what their values are in that creative work whether it be a sonnet, a novel, a short story or a computer program.  One of the scary things about doing creative work is that it always reveals something about you.   (2)

Like all creative works, one can work at many levels, and one can also employ irony, sarcasm and so forth. Thus one may have to have two neurons to rub together to realize when reading Jonathan Swift's “A Modest Proposal...” that he does not actually think that people should eat Irish babies to deal with the overpopulation problem. Similarly one may write bad software to make a point, or simply because the client wanted it to be bad, as they so often do.

Now we have one more point to make. Those who are not lucky enough to have been rewarded for their work and have to get a new career in todays “great recession” have an overwhelming number of new, but not altogether well designed, frameworks, operating systems, languages and so forth to learn. When they get a job, the problem becomes much easier as long as they hold that job. Android is certainly one of those things one might learn to try and be employable. Thus when it is badly designed or deliberately difficult to use, it is particularly frustrating to those of us who are not being paid.

So my response to the anonymous commenter who tells me that I had better revisit the subject but doesn't have the guts to sign his opinion nor the time to explain why this might be so is that I would be happy to. My rate is $4K/day with a 10 day minimum due up front.

Let me know when you want to start.

________________________________________________

1. Which may, unfortunately, be the case.

2. There are two jokes related to this "the work reflects the personality of the author".  The first is a film school joke: everyone's first film is about sex, whether they know it or not.   The second is a joke from the field of industrial organization: that an organizations building (headquarters, usually) tells us something about the company.   Thus the Pentagon tells us something about how the DOD works, and the building of the American Museum of Natural History tells us something about how the museum organizes itself.  This is one of those jokes that has an awesome amount of truth to it.