Draft
A friend, who knows computer science better than I ever will, declined to comment on the usefulness and validity of machine learning to me. Actually, I had not asked him about that, at least I didnt think I did, but maybe I did without realizing it. Its quite possible. Anyway, he declined comment, sensibly pointing out that he was putting his energy into areas where he felt he had some understanding and intuitive feel for what was going on and what needed to be done.
One more time, I envy my friend's lifestyle. Out here in the so-called real world (it isnt very real, just not the world of academia) one has no choice but to have an opinion on this topic.
And speaking of the real world, I would tell anyone who wants to use "Machine Learning" in an application to be aware that you will need a lot of time and the ability to possibly fail, especially if this has to work as a useful application vs a proof of concept.
Thursday, December 20, 2018
Wednesday, December 19, 2018
GANs That Work and GANs That Fail
draft
So after the previous "circle test". I try two more tests using the exact same GAN networks, batch size, etc. One is a "square test", similar to the "circle test" and one is a "grid test" of 100 variations on a grid. See examples of both below.
And here are some results from the "square test"
I think these are looking pretty good.
But now when we get to the grid test, well, not so good.
This makes me wonder if GANs can really handle anything other than very clear graphic shapes. Or maybe a completely different type of network is required.
So after the previous "circle test". I try two more tests using the exact same GAN networks, batch size, etc. One is a "square test", similar to the "circle test" and one is a "grid test" of 100 variations on a grid. See examples of both below.
And here are some results from the "square test"
I think these are looking pretty good.
But now when we get to the grid test, well, not so good.
This makes me wonder if GANs can really handle anything other than very clear graphic shapes. Or maybe a completely different type of network is required.
Dershowitz is a Right Wing Asshole
draft
Dershowitz thinks it is just groovy to put a rapist on the Supreme Court.
https://thehill.com/opinion/judiciary/417849-justice-kavanaugh-should-not-be-impeached-or-investigated
Dershowitz thinks it is just groovy to put a rapist on the Supreme Court.
https://thehill.com/opinion/judiciary/417849-justice-kavanaugh-should-not-be-impeached-or-investigated
Why the Space Force is Stupid
draft
The Space Force idea is another attempt by Trump to put his rancid stink on something in an overt and stupid way. The reason for having a "space force" is to try to deal with several different problems in executing a coherent space strategy. Another service is not necessary, some sort of reorganization probably is. This is all made more complicated because so much of what we do in space is not just secret, we can not even talk about it at all. Too many fingers in the pie(s), and too many other agendas (e.g. the Air Force has a lot on its plate, how can it demand more money for space when it does not (in its opinion) get enough money for air). Its not exactly a mess. A separate service is probably not needed at this time.
The Space Force idea is another attempt by Trump to put his rancid stink on something in an overt and stupid way. The reason for having a "space force" is to try to deal with several different problems in executing a coherent space strategy. Another service is not necessary, some sort of reorganization probably is. This is all made more complicated because so much of what we do in space is not just secret, we can not even talk about it at all. Too many fingers in the pie(s), and too many other agendas (e.g. the Air Force has a lot on its plate, how can it demand more money for space when it does not (in its opinion) get enough money for air). Its not exactly a mess. A separate service is probably not needed at this time.
Letter to Rep Carbajal (CA 24) About the Latest Kavanaugh Abomination
draft
My letter to Rep Salud Carbajal of CA 24 is as follows:
My letter to Rep Salud Carbajal of CA 24 is as follows:
The recent dismissal of ethics complaints against Kavanaugh is an abomination. Why does our government think it can keep doing things like this and still have any credibility? I have had it with this right wing garbage dominating our government. It is time for my elected representative to speak out, loudly, now and often.
As far as I am concerned your only job in Washington is to impeach Trump, Pence and Kavanaugh (and possibly Gorsuch).
You are my representative, please speak out, now.
Thank you.
Michael Wahrman
Tuesday, December 18, 2018
Incremental Notes on GANs
draft
So there are a variety of things I have learned about GANs in the last few weeks.
1. I am limited by my GPU memory. The more memory you have, the more you can fit into the GPU, the larger the batch size. A batch size of 6 may have very different results from a batch size of 2, or a batch size of 50.
2. It is only the GPU (in my case a 1080 with 8 GB memory) that makes this at all practical or possible. I have 8/16 pretty good processors here and they can run for 10 minutes and not do a single batch, whereas the GPU can do a batch in 60 seconds or less.
3. 256 resolution is much more practical than 512 resolution.
4. It is not at all clear whether GANs can do any of the simplistic projects I wanted to do with galaxies, nebulae or march posters.
5. With the simplistic test patterns, at least 150,000 samples are required.
So there are a variety of things I have learned about GANs in the last few weeks.
1. I am limited by my GPU memory. The more memory you have, the more you can fit into the GPU, the larger the batch size. A batch size of 6 may have very different results from a batch size of 2, or a batch size of 50.
2. It is only the GPU (in my case a 1080 with 8 GB memory) that makes this at all practical or possible. I have 8/16 pretty good processors here and they can run for 10 minutes and not do a single batch, whereas the GPU can do a batch in 60 seconds or less.
3. 256 resolution is much more practical than 512 resolution.
4. It is not at all clear whether GANs can do any of the simplistic projects I wanted to do with galaxies, nebulae or march posters.
5. With the simplistic test patterns, at least 150,000 samples are required.
CBS Eye Recreated by Machine Learning
draft
Kids, dont try this at home. These machine learning generative adversarial networks are for adults, you could hurt yourself. There are finally some decent results from a GAN on a pattern simple enough to work. But this tells me almost nothing about what the technique can or cant do. This test is very, very simple.
An example of the input images.
And some of the generated images after training on about 125,000 different images.
Kids, dont try this at home. These machine learning generative adversarial networks are for adults, you could hurt yourself. There are finally some decent results from a GAN on a pattern simple enough to work. But this tells me almost nothing about what the technique can or cant do. This test is very, very simple.
An example of the input images.
And some of the generated images after training on about 125,000 different images.
Monday, December 17, 2018
GANs, Machine Learning & Learning on the Internet
draft
By using a simple test pattern and pushing through latest attempt at a GAN, I was able to confirm some suspicions I had about what was going on. And through this I was able to use the right words in my Google search and come up with some posts and references to papers to help explain what was going on.
What I saw in my tests is that indeed my GAN was at various times doing a respectable job at recreating the image class. And then it would seemingly lose its mind, or lose the thread, and go off into its own zone from which it would occassionally return.
The process has been productive and I have learned a lot, but the process is also suboptimal. Its hard to tell from the Internet at first glance whether or not many of the posts that are thrown in your face have been written by people who actually understand what is going on. My opinion at this point is that they do not.
In particular, what 98% of these posts about GANs did not tell me was that this very promising technique is not at all cut and dried. That there are many questions and issues about pretty much every phase of the process, about what type of loss function, about what type of optimizer, about how to avoid "mode collapse" and many other critical issues.
So GANs are an exciting work in progress, a true research project, and unlike many other areas that someone from Computer Graphics might expect, nothing is cut and dried here. It may work in one case but not another.
In my case, it truly makes progress towards the goal, but then it wanders off.
Here is a link to the best post I have found so far on this topic by Jonathan Hull. He has written a dozen or so posts on GANs and so you might want to poke around and see more of what he has written.
https://medium.com/@jonathan_hui/gan-why-it-is-so-hard-to-train-generative-advisory-networks-819a86b3750b
Here is one of the input images as a reminder of what I am aiming for and then a few selected images of results that shows it is on the right track. Remember, our GAN does not know a circle from a politician, so when you see something like a circle that is significant.
And here are some selected outputs from the GAN.
And then here are some images from when it has apparently lost its mind.
By using a simple test pattern and pushing through latest attempt at a GAN, I was able to confirm some suspicions I had about what was going on. And through this I was able to use the right words in my Google search and come up with some posts and references to papers to help explain what was going on.
What I saw in my tests is that indeed my GAN was at various times doing a respectable job at recreating the image class. And then it would seemingly lose its mind, or lose the thread, and go off into its own zone from which it would occassionally return.
The process has been productive and I have learned a lot, but the process is also suboptimal. Its hard to tell from the Internet at first glance whether or not many of the posts that are thrown in your face have been written by people who actually understand what is going on. My opinion at this point is that they do not.
In particular, what 98% of these posts about GANs did not tell me was that this very promising technique is not at all cut and dried. That there are many questions and issues about pretty much every phase of the process, about what type of loss function, about what type of optimizer, about how to avoid "mode collapse" and many other critical issues.
So GANs are an exciting work in progress, a true research project, and unlike many other areas that someone from Computer Graphics might expect, nothing is cut and dried here. It may work in one case but not another.
In my case, it truly makes progress towards the goal, but then it wanders off.
Here is a link to the best post I have found so far on this topic by Jonathan Hull. He has written a dozen or so posts on GANs and so you might want to poke around and see more of what he has written.
https://medium.com/@jonathan_hui/gan-why-it-is-so-hard-to-train-generative-advisory-networks-819a86b3750b
Here is one of the input images as a reminder of what I am aiming for and then a few selected images of results that shows it is on the right track. Remember, our GAN does not know a circle from a politician, so when you see something like a circle that is significant.
And here are some selected outputs from the GAN.
And then here are some images from when it has apparently lost its mind.
Sunday, December 16, 2018
Test Patterns for Machine Learning
draft
I am slowly reducing my test patterns for machine learning to the simplest form that will tell me something.
When this is a little further along, I may package them all up and put it on kaggle as a lesson for the others.
I should mention that these images have been through the Tensorflow/Keras data augmentation process, and are hence smashed in a variety of ways. Here is an example of an original image as created via Nuke. Still not perfect, but better. For what I am doing, it really doesnt matter.
I am slowly reducing my test patterns for machine learning to the simplest form that will tell me something.
When this is a little further along, I may package them all up and put it on kaggle as a lesson for the others.
I should mention that these images have been through the Tensorflow/Keras data augmentation process, and are hence smashed in a variety of ways. Here is an example of an original image as created via Nuke. Still not perfect, but better. For what I am doing, it really doesnt matter.
Shut the Government Down Now
draft
I am perfectly OK if they shut the government down until Trump/Pence is gone, the rapist is off the supreme court, and so forth. I will vote against any representative of mine who permits this travesty to go on even one more day. Its the moral equivalent of the hunger strike in the face of gross injustice.
I am perfectly OK if they shut the government down until Trump/Pence is gone, the rapist is off the supreme court, and so forth. I will vote against any representative of mine who permits this travesty to go on even one more day. Its the moral equivalent of the hunger strike in the face of gross injustice.
Subscribe to:
Comments (Atom)