draft
With the help of my mysterious admirer, Ken Perlin and Supermicro support (thank you BN!), I am slowly building out my little lab here. Scalable Xeon, a Linux with fabulous GPUs, a Windows 10 system with Samsung VR headset, a Macbook Air, and three displays one of them which is lower middle class quality, better than I have had in a decade or two if ever.
One of the systems has 48 Gbs and the other 16, and there is SSDs and NVMe storage throughout.
It is astonishing how much better my work is with decent resources.
But there is a gotcha and it makes me laugh.
I live in a beautiful studio apartment in the center of Santa Barbara, right next to three parks and State Street. Its a garden. Across from the University Club. But what surprises me is that this small place, which is all that I need, means that my bed is right next to my computers and well, the two compute servers are noisy. Sometimes it matters and sometimes it doesnt.
Yes earplugs can help, up to a point.
Strange what a few fans will do.
Showing posts with label infrastructure. Show all posts
Showing posts with label infrastructure. Show all posts
Tuesday, January 15, 2019
Tuesday, November 13, 2018
Lessons on the Path to Righteousness and the Installation of Tensorflow on Centos
draft
Tensorflow is one of the open source solutions to a machine learning back end. That, with a Keras layer on top, is one of the more popular machine learning environments out there. Among other things, it supports both central and graphics processing on most operating systems.
As in so many things in life, a clever or lucky choice can achieve a goal with no effort, but a similar choice can result in weeks, years or even decades of hell.
There are a number of surprises involved in installing these packages on your operating system of choice, and this note is intended to help you, readers, avoid shooting yourself in the foot or the head as the case may be.
1. Never, never, never try to install from source no matter who advises you to. It is perfectly possible to install from source on a bare metal machine without any virtual environments, or you could just hit yourself with a large hammer for a few weeks. Who knew that there were so many different ways to install Python, or that there were so many Pythons? And that is just the tip of a very nasty set of icebergs.
2. So whenever you are given an opportunity to isolate yourself from the real world by using a virtual environment, whether in Python or anywhere else, take it. In particular, for the Windows 10 version, a choice of the python virtual environment and a precompiled version of Tensorflow/Keras will result in a cpu only version in an afternoon. For some of you, you are done and can move on.
3. For those of us in Linux world, you now have to choose between a few specific versions of Ubuntu and everything else. You who would compromise your integrity and have no aesthetic sense are welcome to use Ubuntu. Go, it is there for you.
4. For the rest of us who might use an adult version of Linux, my operating system of choice is Centos / RHEL 7.5 which is the most recent version. I thought I had to compile from source, but this turns out not to be the case. What turns out to to one of the best paths through this jungle is to use the Docker (container) version as follows.
5. Install Docker by registering as a free user of the Community Edition. Having registered, and installed the preferred package from the preferred repository, you are now eligible to use containers that have been registered with Docker.
6. Tensorflow creates a new version more or less every day in a variety of flavors (no GPU, GPU, etc) and puts them out on the Docker registry with such adjectives as "latest" or "stable" for example.
7. Using these magic words you can create the name of a container you want to run. You use one of the magic containers and it loads that part of the container whose layers are not already local, and if you so specify, you are in a shell, in a container, in which you can go into python, load tensorflow and keras and you are off to the races in a cpu version of Tensorflow.
8. Of course, at this point you are now using containers and you will need to spend a day learning about container file systems and other nuance. Its not too bad though.
9. For those of you who foolishly also want to use GPU acceleration, you have chosen a slightly more difficult path. You will have to install a different version of the "docker" program from Nvidia and Github. But once you do, and once you install the GPU driver on your Linux (a bird of a different feather) you can now use a container with GPU from that list mentioned above.
Good luck!
Tensorflow is one of the open source solutions to a machine learning back end. That, with a Keras layer on top, is one of the more popular machine learning environments out there. Among other things, it supports both central and graphics processing on most operating systems.
As in so many things in life, a clever or lucky choice can achieve a goal with no effort, but a similar choice can result in weeks, years or even decades of hell.
There are a number of surprises involved in installing these packages on your operating system of choice, and this note is intended to help you, readers, avoid shooting yourself in the foot or the head as the case may be.
1. Never, never, never try to install from source no matter who advises you to. It is perfectly possible to install from source on a bare metal machine without any virtual environments, or you could just hit yourself with a large hammer for a few weeks. Who knew that there were so many different ways to install Python, or that there were so many Pythons? And that is just the tip of a very nasty set of icebergs.
2. So whenever you are given an opportunity to isolate yourself from the real world by using a virtual environment, whether in Python or anywhere else, take it. In particular, for the Windows 10 version, a choice of the python virtual environment and a precompiled version of Tensorflow/Keras will result in a cpu only version in an afternoon. For some of you, you are done and can move on.
3. For those of us in Linux world, you now have to choose between a few specific versions of Ubuntu and everything else. You who would compromise your integrity and have no aesthetic sense are welcome to use Ubuntu. Go, it is there for you.
4. For the rest of us who might use an adult version of Linux, my operating system of choice is Centos / RHEL 7.5 which is the most recent version. I thought I had to compile from source, but this turns out not to be the case. What turns out to to one of the best paths through this jungle is to use the Docker (container) version as follows.
5. Install Docker by registering as a free user of the Community Edition. Having registered, and installed the preferred package from the preferred repository, you are now eligible to use containers that have been registered with Docker.
6. Tensorflow creates a new version more or less every day in a variety of flavors (no GPU, GPU, etc) and puts them out on the Docker registry with such adjectives as "latest" or "stable" for example.
7. Using these magic words you can create the name of a container you want to run. You use one of the magic containers and it loads that part of the container whose layers are not already local, and if you so specify, you are in a shell, in a container, in which you can go into python, load tensorflow and keras and you are off to the races in a cpu version of Tensorflow.
8. Of course, at this point you are now using containers and you will need to spend a day learning about container file systems and other nuance. Its not too bad though.
9. For those of you who foolishly also want to use GPU acceleration, you have chosen a slightly more difficult path. You will have to install a different version of the "docker" program from Nvidia and Github. But once you do, and once you install the GPU driver on your Linux (a bird of a different feather) you can now use a container with GPU from that list mentioned above.
Good luck!
Sunday, June 24, 2018
Infrastructure and Other Annoying Topics
There are several issues here and they are pretty annoying, they all come from poverty. A short version says that I have several different needs and not all fit any one computer. At the end of the day it seems to be something like this: a laptop capable of driving 4K graphics in real time (think going on location to a planetarium dome), a server capable of serious rendering and some machine learning, a display for reviewing images with some color integrity (not much, but some, see Eizo CS2730), a display for previewing 4K real time and doing development, a half dozen or more (or maybe just one) radio/computer controlled drones for testing, with associated infrastructure, a projector for a variety of purposes in public display and projection mapping, decent camera and lens, and the ability to attend perhaps 3-4 (?) conferences or shows, some of them international.
It is going to be tough. I am way out of date on some topics, less so on others. I have tentatively determined to try and work my way through maybe a half dozen subjects, all based around a text or two per subject, probably drawn from the texts used in entry level CS PhD programs at (for example) a place like Stanford CS. The Pharr book on physically based rendering is one such book. At the very least, a book/course on machine learning, presumably fundamental algorithms, and some others. Simulation of cities, society, and late-stage predatory capitalism? Computational biology? Original sin and the Fall from Grace? Modern sensors and explosives in public places? Kayaking? Burning Man?
"A man alone is easy prey", said Clint Eastwood in one of his movies. He went on to say that "a good man always knows his limitations", but in context it is meant as an insult.
It is going to be tough. I am way out of date on some topics, less so on others. I have tentatively determined to try and work my way through maybe a half dozen subjects, all based around a text or two per subject, probably drawn from the texts used in entry level CS PhD programs at (for example) a place like Stanford CS. The Pharr book on physically based rendering is one such book. At the very least, a book/course on machine learning, presumably fundamental algorithms, and some others. Simulation of cities, society, and late-stage predatory capitalism? Computational biology? Original sin and the Fall from Grace? Modern sensors and explosives in public places? Kayaking? Burning Man?
"A man alone is easy prey", said Clint Eastwood in one of his movies. He went on to say that "a good man always knows his limitations", but in context it is meant as an insult.
Wednesday, March 5, 2014
When All Roads Lead to Rome
A friend has just finished a chapter of his life and is making a pilgrimage to Rome as a reward of sorts. My friend has tastes that lean towards the exotic and the esoteric. He has studied well the ancient mysteries and is a worthy recipient of the ancient knowledge. He has asked me what he might see when in Rome and I have come up with a tentative first list that does not begin to be exhaustive. It is also a little terse and will hopefully be somewhat annotated and extended later.
Before you go to Rome, buy the Oxford Archaeological Guide to Rome.
Review the website listed below which is about underground Rome. http://www.romasotterranea.it/homepage.html
Before you go to Rome, buy the Oxford Archaeological Guide to Rome.
Review the website listed below which is about underground Rome. http://www.romasotterranea.it/homepage.html
When in Rome, do these things:
Remember when you are there that the accursed Christians stole from everyone and desecrated the sacred buildings that were left in their trust. Whenever you see ancient concrete you are seeing the foundations of a building that has had its stone and marble exterior and other accessories, windows, doors, lamps and so forth, stolen.
Remember that there have been styles of restoration over the last few millennia and that it can be very hard to tell without guidance how old certain things are, what is a restoration, what is authentic etc.
Some of the most hated architecture is from the Fascist period of the last century. But this posed hatred is an affectation and history will probably judge their work better than the art snobs of a few years ago. Or maybe not, I am just biased against art snobs in principle and haven't really looked that closely.
When the Roman Empire "fell", it did not actually fall and lasted another 1000 years or so in the East, and that nothing architectural really went away. The population dived from a high of about one million to a number that is quite small, quite possibly a few tens of thousands.
The entire city then was a ruin and a garden. As time went by, some of the rich families acquired great tracts of Rome and made them their private gardens. And why not? No one else was taking care of things. If only they had not destroyed so many antiquities in the process of making their gardens. See the Farnese Gardens here at Wikipedia.
Remember that Rome is not open all the time, things have weird hours, sometimes an attendant or guard will let you in if you ask nicely or perhaps provide a tip or honoraium. Apparently you need to make a reservation in advance to see the Vatican. Same with the Borghese gallery which comes highly recommended.
Many of the originals of things you see are in museums and what is in situ, on the street as it were, is a restoration.
See the Museum of Roman Civilization which has among other things a recreation of the 2nd century Rome from the Forma Urba..
Count how many words in this post are standard English and yet are also perfectly good Latin.
See the Museum of Roman Civilization which has among other things a recreation of the 2nd century Rome from the Forma Urba..
Count how many words in this post are standard English and yet are also perfectly good Latin.
Climb the victory tower of Trajan which inside has a spiral staircase to the top.
See one of the vast caverns inside the hills of
Rome left over from quarrying the local tufa before they built their monuments out of marble, an affectation they picked up from the Greeks. I keep reading about these caverns but I have never heard of anyone who has actually seen them. Possibly they are closed or just dangerous.
Constantine giving the "finger"
When you see a giant marble head, or hand or foot, recall that Romans often made their cult statues (the image of the God for the temple) in a way that economized on the marble or other stone required. The head, hand, and feet are what was exposed of the statue, the rest might be in a toga and therefore did not have to be carved (not to mention quarried, transported, etc).
See the black stone from the 6th century BC and ponder the meaning of the archaic Latin.
Walk the floor of the original Roman senate (not the one that Augustus built) and stand where Julius stood when he was murdered by his fellow senators in the name of freedom, which really meant to preserve the privileges of their class.
See the aqueduct switching center and
distribution system.
Find the recently discovered Orbs of Imperium hidden by Maxentius when he was defeated by the traitor Constantine at the Milvius Bridge.
Horatius at the Bridge. Find the bridge.
Make contact with those who are
attempting to revive the old religion in Rome against the hated
Christians and perform some ceremony with them.
Go to the Kings House (the Regia) and
ponder what it means about the origins of Rome, the kings of Rome and what is true and what is not.
Go to the bridges of Rome and remember
that the leading religious figure was the Pontifex Maximus and that
bridges and early Roman religion are somehow connected. (The Pope
is still officially the Pontifex Maximus of Rome).
Go to the House of the Vestals and
light a fire.
Go to Alba Longa and wonder if the
brother-murderer Romulus really came from there.
Trace the route of the Lupercalia. Best to do so naked while wearing a thong made of the sacrificed
goats or dog. The Lupercalia was probably an initiation rite of young men to a brotherhood from the time of the earliest Rome or before.
Trace the route of the Triumph.
Go to where the Sibylline books were
kept (in the archives of one of the Temples) and lament the
loss of important knowledge in the various fires and tragedies of Rome.
Go underground at one of the Baths and
see how the plumbing worked.
Go to Ostia / Portus and see the port
of Rome which has much more of an authentic Roman city from the late
empire.
Go to Pompeii/Herculaneum before they are destroyed by being exposed to the weather and the light and ask yourself why the roads had those stepping stones.
Go to the tombs and memorials of the
murdered Gracchi Brothers and learn about the Social Wars and then think about the future of America.
Find the Milvius Bridge and realize that this is where Western Civilization was destroyed by Constantine who fell into superstition
and began to worship the hateful murdered god/king of the Christians.
See the Parthenon and imagine what it
looked like before the wretched Christians got there. It has stood for 2000 years and we can't make buildings that last for 100.
Tour some of the catacombs and realize that they were not just for Christians and that Christians never really worshipped there in secret. The catacombs were a response to the lack of space in and around Rome and its expense. So they dug underground and put their crypts there. Many of the catacombs have not been explored (or at least we are told that).
Realize that the Romans were not permitted to bury
the dead within the sacred boundaries of the city (the Pomeranium) so
they built their tombs on roads leading out of the city. Therefore
go to the Appian way and outside the formal walls of Rome see some of
the tombs. In a prime spot outside the walls of the city is the tomb of the Scipio Family, the family of the famous Scipio Africanus.
When in Rome keep your eyes open for the family name Colonna. I am friends with Kerry Colonna who worked with us at deGraf/Wahrman. The Colonna mansion and private art collection, one of the three most prestigious private art collections in the world, is open for tourists on Saturday morning each week. Go see it.
When in Rome keep your eyes open for the family name Colonna. I am friends with Kerry Colonna who worked with us at deGraf/Wahrman. The Colonna mansion and private art collection, one of the three most prestigious private art collections in the world, is open for tourists on Saturday morning each week. Go see it.
See the tomb of Augustus and imagine what it looked like before the Christians plundered it. (Note: I read that the tomb will have some restoration work done to it after all these years of being essentially ignored. Something to do with the 2000 year anniversary of the death of Augustus).
See the Golden House of Nero underneath one
of the Christian Slave Churches.
See the secret library of the Vatican where
both truth and lies are told.
Visit the Etruscan tombs. What were the "mirrors" for?
See the great sewer of Rome, the Cloaca Maxima, built in the time of the Kings. A good article on the history of the Cloaca Maxima can be found here.
The Cloaca Maxima was originally above ground and used as drainage for rivers that flooded what would become the Forum. It was started in the 6th century BC and many workers were killed building it.
See the great sewer of Rome, the Cloaca Maxima, built in the time of the Kings. A good article on the history of the Cloaca Maxima can be found here.
Remember that the Cult of Mithra was a
a late empire cult, mostly in the army. Perseus slaying the Bull
probably refers to the secret information involving what the soul
must do after death to pass safely among the stars. This Mithra may
or may not have anything to do with the religions of the East,
although they are certainly an Eastern import.
Read about the time I first saw a Roman ruin in this post.
Friday, March 29, 2013
The Future of Decay: The Abandoned Tunnels of the PA Turnpike
"Always look on the bright side of
life" the crucified thief advised Brian as he was nailed to the
cross. Even as America declines into impotence and decay, led by
corrupt and incompetent leaders, engaged in hideously expensive wars
at the behest of morons and torturing the natives, working with
diligence to disenfranchise workers, destroy unions, and send jobs to
China who have in the last decade executed the largest espionage
program in history against us, there are still things to be proud of
in America.
As the country declines and collapses
the bright side is that infrastructure is abandoned and these
fascinating and dangerous artifacts of our former civilization can be
repurposed as tourist attractions. From old missile silos, to
airports, from secret bases to abandoned tunnels, roads, factories
and mills, America gains new potential theme parks and sources of
revenue.
America may never rival the great
centers of decay such as the former Soviet Union, but it can still
hold its own and contribute our own uniquely American tradition of
decay, corruption and degradation.
Forget "Tomorrowland" and
look to the decaying past to see the future of America.
The first stop on our tour is the
abandoned tunnels of the Pennsylvania Turnpike in Eastern
Pennsylvania.
The Pennsylvania Turnpike was an early
toll road in this country connecting Pittsburgh and Philadelphia, and
extending 360 miles across the state, east to west. The turnpike
utilized seven abandoned railroad tunnels built in the 1880s. These
were dual lane tunnels, one lane in each direction. As time went by,
the single lane through the tunnels became a bottleneck and caused
major congestion. Either new tunnels needed to be built, or the
tunnels themselves bypassed. Of the 7 tunnels, 4 were expanded by
building a parallel tunnel to allow for two lanes in each direction,
and 3 tunnels were abandoned and a new section of the turnpike built
to go around the obstacle rather than through it.
Like the WW2 German Submarine Basers in
France these tunnels were too expensive to be demolished, but unlike
the submarine bases, the Commonwealth of Pennsylvania has sold the
tunnels and the connecting road and access right of way to a nature
conservancy, the Southern Allegheny Conservancy, who has worked to
preserve the area. It is working with "Pike2Bike" a group
which is working to make part of the abandoned turnpike into a bike
path.
Our Host
See thie following video for a tour of
the tunnels:
A web site on the abandoned turnpike:
The Pennsylvania Turnpike on Wikipedia
The Abandoned Turnpike on Wikipedia
Subscribe to:
Posts (Atom)