Linux AMD 64 neural net

Linux specific questions.
greenink
Posts: 188
Joined: Jan 28, 2016 15:45

Linux AMD 64 neural net

Postby greenink » Jan 02, 2017 15:37

I had a choice of continuing my neural network code in Java to make it more generally available or fully optimizing the code in Linux AMD 64. Java's getting kinda old etc. So I'll try with whatever suits the hardware I have best. With a graphics card I could get about a 50 to 100 times speed improvement, but I'll just work with what I've got:
https://drive.google.com/open?id=0BwsgMLjV0BnhSGN0NEZ2QkxoOWM
I'm going to do some experiments with gradual feature learning and some other ideas that present themselves.
greenink
Posts: 188
Joined: Jan 28, 2016 15:45

Re: Linux AMD 64 neural net

Postby greenink » Jan 04, 2017 5:54

I've added two layers of gradual (data driven, unsupervised) feature learning to the network followed by 1 read out layer for each color. I'll see if it does anything interesting.

https://drive.google.com/open?id=0BwsgMLjV0BnhR2RTWURlZ3NuUjQ

In any event the other idea I have at the moment is to evolve a few simple feature layers and then follow that by a read out layer. The seems a simpler task than trying to evolve a - full on - deep neural net. Yet introducing enough smarts to be able to do something worthwhile. I also have an idea about (greedy learning algorithm based) decision/prediction trees and reservoir computing, I think you could do something out in the aether with that. There is the idea of randomly switching between trees trained on similar data to prevent repetitious loops in long prediction chains. And if you create a number of different chains you could view those different chains as different plans for the future, and decide the best one.
greenink
Posts: 188
Joined: Jan 28, 2016 15:45

Re: Linux AMD 64 neural net

Postby greenink » Jan 04, 2017 16:02

Slightly better organization:
https://drive.google.com/open?id=0BwsgMLjV0BnhcHpPNDNFVEtBNEk
Two layers of gradual feature learning and then one read out layer for each color.
I'll experiment around with it tomorrow some more. It does look like it can produce moderately rich results while using only local learning (no back propagation.)
greenink
Posts: 188
Joined: Jan 28, 2016 15:45

Re: Linux AMD 64 neural net

Postby greenink » Jan 05, 2017 22:53

The dot product features I used are far too selective especially with small training sets. I'm experimenting with just a single feature layer at the moment. Anyway I want to move on and experiment with evolution and decision/prediction trees again.
greenink
Posts: 188
Joined: Jan 28, 2016 15:45

Re: Linux AMD 64 neural net

Postby greenink » Jan 15, 2017 13:07

I went back to using FB class types rather than using types with namespaces. The code is far neater that way. However I did stay with using pointers rather than FB arrays in general. Again that seemed to reduce clutter.
Anyway it is another attempt at evolving neural networks. It seems to be getting somewhere this time. I don't actually expect to get great results on a 2 core CPU with that experiment. It mapping a high dimensional space to a high dimensional space a bit too literally. I'll try using the same net in high dimensional to low dimensional experiment to see if it has any real learning capabilities.

https://drive.google.com/open?id=0BwsgMLjV0BnhQkVXem9XU28xNzQ
greenink
Posts: 188
Joined: Jan 28, 2016 15:45

Re: Linux AMD 64 neural net

Postby greenink » Jan 17, 2017 13:36

I dropped using constructors and destructors and just used subs (init and free). That way I don't make assumptions about things I don't know about and maybe fix a dangling pointer issue, or some other not correct thing I did.
https://drive.google.com/open?id=0BwsgMLjV0Bnha285SVRpRW02V1E

The next this is to move on to a more sensible experimental setup. That is high input dimension to low output dimension a see if the thing can really find intelligent solutions.

Return to “Linux”

Who is online

Users browsing this forum: No registered users and 1 guest