Reading List on Data Storage for the Computer Illiterate

This is the second post in the boring “build a backup for your studio” series of posts. The first post is here.

The primary reason I am writing these boring posts is the occasion of having a friend of mine, a professional photographer, recover from a catastrophic data failure.  Whenever I would bring up terms like "network file server", she would put on that expression of "I am just a girl and I dont know what that means" that so many of us are so familiar with.   The good news is that even my brilliant professional photographer friend can pick up these computer terms with very little effort.

This stuff is not hard to understand.  What is hard to understand may be how things are implemented to work well, if indeed they do work well, but the basic concepts are straightforward.

The fact is that most professional users of computers, even those in their own home office or studio, will have a heterogeneous collection of files that look like they are all attached to the local computer even though they are not.  Some OS's handle this better than others "out of the box" but they all accommodate it.

Most of the time you, the user, do not care if a file is local, or on your local network, or even further afield. But you very well might care if you are your own systems administrator or your studio architect and since most of us are our own administrator, you have to know this stuff.

So get over your computer anxiety and gender bias and get this done.   Here is your Wikipedia (and one optional Dell white paper) reading list.

1. All your files on a computer is managed by a file system.

https://en.wikipedia.org/wiki/Computer_data_storage

https://en.wikipedia.org/wiki/File_system

2. Most simple storage on your basic home computer is directly attached storage.

https://en.wikipedia.org/wiki/Direct-attached_storage

3. All modern computers today also support network attached storage.

https://en.wikipedia.org/wiki/File_server

https://en.wikipedia.org/wiki/Network-attached_storage

4. Whether your storage is direct or on your local network, there are a variety of techniques designed to take these relatively cheap disks for personal use and make things more reliable. There are a variety of ways of doing this. The simplest is disk mirroring. RAID is a way of formalizing some of the existing techniques of combining multiple disks into a more reliable, or better performing, “virtual” disk.   You mostly only care about RAID 0, RAID 1 and RAID 5.

https://en.wikipedia.org/wiki/Disk_mirroring

https://en.wikipedia.org/wiki/RAID

5. RAID can be implemented in hardware or software or both. People used to care passionately about which one they had, hardware or software. The reality is that you should not care which one it is as long as it is reliable, fast and low maintenance.  For those who think they care, here is a Dell white paper on the topic (optional).
ftp://ftp.dell.com/app/3q03-Dum.pdf

6. But a file system, or a file server, or a reliable disk subsystem is not the same as having a backup system, although it may be a part of that system.

https://en.wikipedia.org/wiki/Backup

Now we can get on with the exciting yet boring design of our backup system.

Is a Worker With Baggage Like a Plant That Has Bolted?

In this post, I want to ask the deliberately limited question of whether or not a “more experienced worker with baggage” is similar in some ways to a plant that has bolted.

First lets define our terms. To a human resources person, a potential employee with baggage is someone who has accumulated behavior, ideas, concepts and so forth during his or her previous employment or personal life which might complicate their working smoothly in the present situation. This is my impression of what the term means, it is not a formal definition from a human resources guide.  God only knows what these guardians of corporate propriety think.

On the other hand, everyone who has had a garden know what “bolted” or “to bolt” means. It means that for one of a variety of reasons,  usually water stress or change of season but sometimes just maturity, a plant enters a different stage of life that is usually not very useful to the gardener.  Generally, the plant prepares to get the hell out of there by preparing to generate seeds. It is very similar to the concept of “going to seed” and it implies that the previously useful plant is now nearly useless unless of course the goal is to generate seeds. Lettuce, which previously was great, is now bitter. Basil, which was amazing, changes flavor to a much lesser form, and so forth.

Is this romaine lettuce about to become unemployable because of its work experience?

In other words, our theory goes, that instead of becoming more colorful, more experienced, more valuable because of things that one has learned that only life can teach you, the potential employee instead appears to be bolted, too concerned about past battles, too filled with preconception about certain types of people or certain businesses, that it interferes with getting the job done with enthusiasm and initiative.

This is an important question because pretty much every interesting person I know who has worked in a field for a while, all of these people have experience that is very real and which will affect their future work. Is that experience positive, or does it make them bolted, or appear bolted?

Furthermore, the judgement, the final judgement of whether the experienced worker unit has value is made, and must be made by people who have neither the experience or knowledge necessary to make a judgement as that might be conventionally thought of.  Rather these keepers of the just and the right have a HR handbook and relevant HR experience to be able to judge.

Sadly the experienced worker comes in with about 10 strikes against them as they have almost certainly been guilty of the unforgivable sin of not making enough money in their previous endeavors.  This is self evident because if they had been successful, defined in the beautifully elegant American manner of accumulating cash, they would not be applying for a job here, but would be in Paris or Bangkok or Manhattan or Aspen managing their certificates of deposit or frolicking with splendid examples of the appropriate gender or genders as the case may be.

The issue of whether experience is the same as baggage, negative and not positive, is just one of the many issues that the meta-concept of baggage brings up.   Can baggage be turned into useful experience through a change in attitude?  Is it fair to attribute baggage to someone without understanding what led to this belief or issue which is now being called baggage?   Is learning from ones experiences baggage?  Is what the human resources person or corporation looking for is not really a person without baggage, but someone who is merely naive?

Indeed, naivete might be the very greatest virtue in a situation like this.  The work output of the virgin is all too likely to be more effusive and extravagant than the work output of the jaded or the sophisticated cosmopolitan who has seen it all.   Those who have not been wounded in love and life are perhaps more likely to go over the top with a rebel yell onto the killing fields and the line of bullets then those who have been there, done that, and knows how much it hurts.

Once a person has this real experience, are they irretrievably "bolted" and unfit for duty in the Globalized Workplace?  Is work experience necessarily a form of disability?

One thing is certain.  In America, business has no responsibility to this wounded and arguably disabled victim of the workplace.  Having fallen in the field of battle but not having the decency of dying and / or going away, he or she degrades themselves by attempting to return to the front lines attempting to fight, that is, to get a job.  Why don't these wounded soldiers just go away and die?  It would be better for them and far less embarrassing.  Business owes nothing to these impoverished survivors.

In America, at least, that much is clear.

Computer Language Preference by Country

Let us say for a moment that I had to get a real job instead of doing what I do now, which is writing a blog, writing little programs for my friend in NYC, reading books and surfing the web, etc.

Although I know (literally) at least 100 (computer) languages, there are only a few of them that I routinely use to "get something done" and which I am comfortable that I know the full extent of that language such that I could be professional at it. It turns out that language use (computer languages) preference differs by country. Here is a map of that use.

So apparently I can work in France and Finland.

Is emigration an option?

Mid Summer 2015 Container Garden Report

This is our mid-summer review on the container garden and the various techniques and choices made.

There has been almost no rain, almost no overcast, and very hot temperatures. Things were different this summer in various ways.

We tried the following techniques (a) put the lettuce and the herbs such as basil in partial shade in the hope they would not bolt so quickly. (b) preventively spray with copper solution and neem oil now and then, (c) when disease or insects attack, spray with various solutions and then ruthlessly and carefully remove the affected areas and / or remove the entire plant, (d) leave more space than ever before between plants, especially the beans and tomatoes, even though this would probably reduce overall yield because of less growing area, (e) provide support for everything, beans, tomatoes, and cucumbers, (f) in the case of tomatoes, try growing the seedlings in these peat moss starting pods that they sell.

All of these techniques worked out to one extent or another and are recommended.

What did not work out was that we had total failure on our carrots and our peas, two different types. I do not know why. I suspect that the peas were victims of the birds, see below.

One surprise was that for the first time, the birds viciously attacked small seedlings, particularly the tomatoes, cucumbers and peas.

Bird countermeasures consist of $20 of green plastic fencing cut half height and surrounding each seedling with a cylinder of fencing material. This worked splendidly and had the additional benefit of providing support to the plant (cucumbers and tomatoes).

The end result was pretty good availability of romaine, green beans and just ok cucumbers. Basil was very useful. Almost all of these but the basil are now history. The tomatoes are just starting. Peas and carrots were a total failure.

We had less disease this year. The green beans always had a lower leaf yellow rot that I removed by scissors. One cucumber plant had a billion aphids, knocked back with insecticidal soap and then the plant removed. The cucumber leaves always had some sort of horrible rot that might have just been leaf burn and which I ignored.

Strangely, plants that in the past had delivered many crops only delivered one this year. Green beans and cucumbers are most notable here.

Going forward, the use of shade for lettuce and herbs, the much greater space between plants, and the use of the green fencing are all solidly recommended.

This cost very little this year, as we are in that sweet spot that equipment bought can be reused but new equipment not needed.

Finally, one final word of caution. If one were to do this to actually live on or for economic purposes, the scale would have to be vastly increased, and no doubt new issues would emerge.

The Boring Topic of Designing a Backup System For Your Studio

[This post was prompted by the occasion of helping a friend try to recover the data from her failed disk server. So the annoying details of this problem, and the necessity of dealing with these issues, is on my mind.]

As part of a series on designing, building and running a small computer animation studio, we are going to have to discuss backups. I will try and break it into small pieces because, frankly, it is a real bore. When we started using computers we did not do so for the joy of making backups which is like taking out the garbage, its not our first choice of how to spend our time. Furthermore, it turns out that there are choices to be made here, and real design issues. I am sorry about that. It just is.

When people started using computers, probably no one told them that they were now expected to be responsible adults about how they cared for their data or run the risk of losing it.   But all of us who have been using computers for a while know this only too well.  You can learn from our mistakes and save yourself a lot of trouble.  

When you drive a car, you are expected to learn how to drive safely. When you work at a real corporation or a University, then it is likely that your professional work is already being carefully backed up and protected, at least to some extent.   But the rest of us, at small companies or on our own, have to put our own system in place.

Keep in mind that hard drives, big or small, solid state or otherwise, are not intended to be perfect.  They have a known failure rate, and even though the manufacturer knows that some of their disks will fail, they only know this on the level of probability.  Disks are made in batches and the failure rate of disks within a batch are estimated as that is part of creating a warranty for the drives.   But disk failure is not the only cause of data loss.  

So here are some basic definitions and principles. In later posts we will go over some of the design choices you may have to make, are likely to have to make, when you design your studio.

For those of you who think I am less creative because I worry about such things, please go fuck yourself. Thank you.

1. The place where you do your professional work might be called your office, or it might be called a studio. A studio can be for one person or 1,000 people. The work might be your personal artwork, or your personal financial records or it might be a very expensive collaborative technology and creative project with a $100M budget.

2. All of these offices and studios need to have given some thought to how much protection they need to give their data in case of disaster, what is the likelihood of disaster, how much it is worth to them to lose one days work, one month's work, one year's work, etc.

3. The goal of a so-called backup system is to provide a level of protection for your data if disaster strikes for any reason, whether by computer malfunction, act of God, or human error.

4. No backup system is perfect, but different backup systems provide different levels of security at different costs, where costs means varying amounts of capital, costs going forward, attention that must be paid to maintaining the system, technical expertise and so forth.

5. A simple backup system well executed is better than a technically complex system that is over the head or beyond the needs of the intended user. An expensive or technically complex backup system that is not well implemented or maintained may be worse than no backup system at all.

6. A backup system is holistic. Together it provides a level of protection.  If some of the pieces work and some do not, you may still have a level of protection.  Thats the plan.   But it is better if all the pieces work generally speaking of course.

7. Backup systems are usually layered, that is, you have more than one protection so that if one fails you do not lose all data, but can fall back to another level. Generally this is implemented as a system to improve the reliability of the main file servers combined with discrete backups saved in a vault from earlier periods.

8. Backup systems are probabilistic. There is a probability of disaster, a probability that any one backup will not be readable. No backup system is perfect, but a good backup system will make the probablility of losing all your data much less likely.

9. Backup systems must be tested before they are used or you run the risk of not finding out that there was a problem until it is too late. This is an extremely common occurrence.

10. No one but you can judge whether this effort, these costs, and so forth are worthwhile. Only you know what this data is worth.

and finally,

11. I have found over the years that I never had too many backups.

In a later post we will go over some fundamental design choices and the kind of risks you will need to protect against.

Lynda Weinman and the Early Days of Computer Animation

For those of you interested in trivia from the early days of computer animation, I have a somewhat interesting story.

When we were founding degraf/wahrman, a variety of people helped us out. One of them was (and still is) a truly delightful and wonderful woman who helped us in dozens and dozens of ways including, among other things, helping us set up our office, helping us set up our finances, and spearheading and completely owning the early use of the Mac for previsualization, in this case for Star Trek V and Ralph Winter, which got everyone a lot of publicity. She was/is also an animator, a friend of many people in animation, and I have no doubt that she was in part responsible for the good vibes surrounding our startup.

She was also from the earliest days a complete believer in the idea that computers such as the Mac could transform peoples lives for the better and enable their creativity. Her idealism motivated everything she did to a remarkable extent. After Star Trek V she had bigger fish to fry and probably most of the people who later worked at dWi did not even realize she had worked there. But she went off and among other things started doing conferences about Flash, and then started an internet company to help people learn to use their computers.

Apparently, a few weeks ago, she sold that company, Lynda.com, to Linkedin for 1.5 billion. It is hard to believe that someone who is so idealistic and so well-meaning would do well in such a practical way, but Lynda Weinman is really that amazing. Anyway, I wanted to publicly congratulate Lynda and thank her again for her help long ago and far away.

Supporting Eccentric Ethnic Self-Identification

I think we should all be grateful to Rachel Dolezal, formerly of the NAACP, for the whirlpool of shit she has stirred up with this whole "identify as black" controversy.    I suspect that she is speaking for a small, but under-recognized segment of the world and American population who identify as something that they are not, at least not exactly.

In particular I want to bring to your attention a certain category of person who identifies as a member of an ethnic group that does not exist, in this case green people, particularly green superheroines, and ask you to give them your support.  As Kermit the Frog so famously pointed out, it isn't easy being green.

I have no idea who this is, but I am sure that she is sincere in her green self-identification.

Whether or not race technically exists from a DNA point of view, there certainly are differences between ethnic groups, differences that are perceived to be very important.  And how noble and self sacrificing that someone should self-identify not only with a small, and no doubt oppressed and misunderstood ethnic group, but actually to identify with a group that does not have any members, at least not on this planet.  At least not as far as we know.

How many such people are there out in the world?    Well, it is hard to say.  Anyone who has attended a science fiction or comic convention can testify that there do seem to be quite a few of these entertaining eccentrics.    I would like to believe that Americans will remember their historical tradition of embracing and accepting diversity and lend firm support to any woman in spandex who wishes to self-identify as She Hulk.  I also believe that we must extend this same privilege to men who also self identify as women who are green, although it is a little harder for me to be as enthusiastic about it, I can certainly agree with the point of view that it is their right.

I hope you will agree with me that our world would clearly be a better place if we had more such people.

At the same time I would like to bring to your attention a cultural resource of great value.  In looking for examples of green she-hulk representations, I noticed that most of them were hosted on a web site called "www.deviantart.com" which is an organization of people who create or collect, well, deviant art.   Its a pretty great collection, no matter what your interest, it probably has a group or two dedicated to it.  Another excellent resource the Internet has enabled.

There are many visions of She Hulk out there

She Hulk on Wikipedia
https://en.wikipedia.org/wiki/She-Hulk

Coming Soon on Global Wahrman

Coming right up on Global Wahrman: an essay on what Baggage means to a Human Resources person and why nearly all, or all, experienced workers have "baggage", an essay on the literary genre of entertainment fiction known as predicting the future, and an essay on America's slide into failure, poverty and submission to the new world leader, Communist China.

Also, in the cyberwar between the US and Russia, we have new news on Russia's paid trolls and the stupid morons in this country who believe and enable them.   My goodness, when you need a stupid American, it isnt very hard to find one, now is it?

Not to mention a post about what Nina Z (Miles G's significant other) told me about middle class values.  Its very interesting, you will want to know this.

And the passing of Tanith Lee and Christopher Lee.

Also, a post on a distant colleague who committed suicide and what a surprise it was. Yeah, right, bullshit. Even I knew she was depressed.  Maybe her death is more about our community's failure to act proactively to help those who are in distress, he wondered, out loud, sarcastically.

Where is Anne Graham of after science when I need her?   I am looking for that pure spirit of inspiration that comes from the untainted.  Anne ?

I appreciate all of you out there who do seem to be reading my blog, even if I do not know who most of you are.  I do wish that you would comment more as I have a feeling your comments would add a lot..

We just ran an unscheduled experiment on my brain by going off all ADHD medication for several weeks, and the results were very surprising.  You see, even though I slept most of the time, and could not do even the simplest tasks that needed doing, in certain very important ways I was very productive, more productive than I normally have been in spite of my reliability and calm demeanor brought on by the ADHD medication. This is a real problem.   What do I want?  Creativity or stability?

Scientists Propose Force Feeding Brains to Right Wing Sufferers of Dementia

Scientists have discovered that a New Guinea tribe that routinely ate the brains of their human enemies not only did not suffer from horrible Prion-caused dementia, but actually developed genetic immunity to this tragic brain-destroying disease.

“Quick action is needed to bring this immunity to other members of the human community”, says Dr. Filbottom of the East Saxon Community Medical Center, who was not an author of the study. “Since clearly our political right wings the world over are suffering from a form of dementia, denying evolution or global warming, by forcing them to eat human brains we can potentially cause them to develop an immunity to their debilitating and life-threatening disease that causes so much hardship and destruction in the world. We must act at once to force feed them brains if necessary in the hope that they will also develop this immunity".

The proposed treatment is to begin with neoconservatives and inject into them mad-cow infected brain particles.  "Just imagine we had this treatment a few years ago, the whole second Iraq war might not have happened at that huge cost of lives and money which did no good.  We must begin this process at once and hope we are in time to save us from other delusions of the right wing demented.  They need our help."

Read more about this exciting discovery here:

http://www.theguardian.com/science/2015/jun/10/brains-helped-papua-new-guinea-tribe

Oh That Free Market

Why bother to write about the "free market" you may ask?  I do this for your own good, so that my peers and other colleagues can have the benefit of my experience and knowledge whether they appreciate it or not, and perhaps rise above their circumstances to which an unkind fate has condemned them.

The nominal reason that I have written these recent posts about the "free market" is that they are a part of a discussion with a friend about how society should set social policies, particularly involving unemployment caused by foreign Government subsidies.  He believes that the "free market" sets the best policies in all cases.  A central point of discussion in this discussion is what is a "free market".  He contends that a "free market" includes obvious laws and morality involving such things as racism, sexism, child labor, the environment, minimum wage and so forth.

But it doesn't, either historically or currently.  Being compelled to follow standards of morality or laws against certain kinds of behavior is, by definition, constraining the "free market".  It is therefore no longer "free".  It is actually much worse than that because those who advocate some rather heinous social policies routinely invoke the holy free market to justify those policies.

The very term "free market" means to let market forces set policy on issues that involve society, commerce and employment. It means that any education you have is bought on the free market by individuals or groups from companies that provide that service.  Those who can not afford that service do not get education.  It means you do not have a legal minimum wage, because the market will set a minimum wage.  It means you do not need laws against sexism because market forces will eliminate sexism because it is inefficient.   It means you do not need child labor laws because market forces will prevent the abuse of children in the workplace.  It means that health care is provided by industry and those who can afford it get it, and those who can not afford it suffer because they can not afford to buy those services on the "free market:".   That is what the "free market" means.

In other words, the "free market" has an implied morality and that morality is nothing more or less than those with the money get what they want and those without do not.

A modified free market, a market that is informed by and controlled with a variety of laws to restrict the abuse of children or outlaw sexism or racism or protect the environment is not, technically speaking, a "free market".  It is a form of market, yes, but not one that is run by pure market forces.   Furthermore, since the "free market", left to itself, has been shown to lead to many disagreeable results, or at least disagreeable to some of us, those who advocate an untrammeled free market do so as a way of achieving their policy goals in these areas.   They use the "free market" as the theoretical justification for rolling back policies or laws claiming it is more efficient and leads to a more productive and fair society.

The result is that the term "free market" has become associated with policies that will mean the degradation of the poor, the disenfranchisement of labor, the willful destruction of the environment, and the restriction of access to the political and justice systems to the wealthy.   Whether that is what the "free market" meant at the time the term was invented or not, these are the implications of the term as used by political groups today.

Recall also that when the term "free market" was invented as an economic philosophy, there was not a body of experience that could readily say "this is what we tried and this is what happened".  But we do have that experience today to make these judgments.

The pure "free market" has no innate sense of morality beyond the morality of cash. But society's own perception of what is moral and what is allowable in a free society is a moving target and changes, sometimes slowly, sometimes quickly.  Market forces will not, a priori, lead to a result that matches society's current understanding of morality and fair play.  Therefore to advocate that it will is disingenuous or simply wrong.

No one argues against a market system to set prices in many circumstances, or to determine, in many of those circumstances where society should put its resources.  Up to a point, that is, as there are many, many exceptions. And therein lies the dilemma, where do you set that point?  Relying on the "free market" to do so is not going to be satisfactory for many Americans, although it may very well be just the ticket for some of them.

And it is ironic, or perhaps just weird, that so many advocates of the "free market" are in the technology industries and seem to be unaware that they are the beneficiaries of very non market forces.  The technology industry as we know it today was financed in large part by our government as part of fighting the cold war.  It was not market forces that invented and nurtured those technologies, not at all, it was the US Government through its Department of Defense, its Department of Energy, its National Science Foundation and a few other agencies.   As those industries matured and became self-supporting the government moved on to other industries and technologies that needed or deserved advanced research money.   In almost no cases is advanced research sponsored by private industry or the "free market".  Not that I am aware of.

 I think that what my friend *may* mean is not the free market, but something like "a market system that is controlled through a system of laws to enforce moral norms as determined through the political system as regards to the environment, the treatment of labor, education and other forms of social welfare, in a system that has gone to great lengths to see that access to justice and the political process is not unfairly weighted to the rich, but that it is equally available to people of all economic and social classes, not just through law, but through practical means."

Its not enough to say equal justice for the legal system if the rich always get off but the poor do not because of being unable to afford equally competent legal support.   Its not enough to say the "market as modified by the rule of law" unless that law is proactive in making violations of morality a crime (e.g. abusing children in the workplace) and is industrious in enforcing the law.

Market forces alone is not a system that will result in a fair or just society and the idea that it would has been discredited long ago.  The refusal to accept the evidence of your eyes does not make the evidence wrong.

One more thing.   There is nothing funny about the political situation in this country, and the gross abuses of the right wing of the political process and their disingenuous and often hypocritical arguments.   It is time for everyone to grow up and figure out when some right wing thug invokes the "free market", what it is they really mean.