Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - Paul

Pages: 1 [2] 3 4 ... 174
Miscellaneous / Re: ARTUR Exmachina
« on: April 20, 2018, 04:17:08 PM »
Yes, that is exactly what I am thinking.  It is interesting that the path to intelligent robots is could end up being backwards from how it is imagined in Sci-fi (where emotions tend to be absent or developed much later)

Miscellaneous / Re: ARTUR Exmachina
« on: April 20, 2018, 03:08:13 PM »
The significations of that paper here, I think, is taking it in context of the amygdala:

The relating that to Figure 2 from the paper:

The amygdala is outputting to the prefrontal cortex (PF), which is a hub in the distributed circuit for semantics, which are grounded in sensory motor activation.  This means the sensory input, and all semantic representations derived from it, and generated motor outputs, are all flavored by emotional context.  The obvious conclusion is that emotional context is a critical part of the sensory-motor circuit and action decisions.

I interpret this to mean that in order to understand concepts like "curiosity" that are important for more efficient reinforcement learning strategies, the above circuit (with its emotional input) need to be understood and part of the model.

Miscellaneous / Re: ARTUR Exmachina
« on: April 20, 2018, 09:29:14 AM »
I'm targeting more of an insect-level intelligence (which I believe is achievable with current technology).  But you correct that curiosity is an absolutely essential element for any autonomous agent.  The agent must have some core motivation to try something different in a recognized context (or try anything at all in a novel context).

Deep Mind (and similar RL implementations) solve this with a diminishing random chance of non-ideal behavior in any given context.  This definitely works, but it has its own problems.  Training requires huge numbers simulations where the agent essentially brute-forces the problem until it reaches a policy that is deemed to be acceptable.  As we recently saw with the Uber self driving car accident, this may not be good enough in situations where human safety is a concern.

My current thinking is that the key to understanding curiosity (which is critical to effectively implement reinforcement learning) requires first understanding emotions.  I recently read an interesting paper by Friedemann Pulvermuller which goes into the relevant biology in nice detail with excellent visualizations.  It appears that emotional flavoring of sensory motor mechanisms is at the core of how the brain establishes a semantic grounding (and thus is a critical part of choosing what actions to take in a given context).

Theory and Discussion / Re: What is ARTUR?
« on: April 20, 2018, 08:29:01 AM »
I've been quite here on the forum for a while (mainly been active on the Numenta forum).  Thought I would post a progress update.

I spent some time on the Reinforcement Learning problem, and linking it with HTM (which still I believe is the best starting technology for the seed AI).  About this time last year I formalized a strategy, which I described in this thread .  For reference, here is a visualization of that strategy:

I've since discovered a couple of problems with this strategy.

One issue is with the reinforcement TP layer.  The basic problem here is that although the pooling layer creates a representation depicting all expected future rewards/punishments, the result is that the system picks a future motor command with the most reward (skipping required previous motor commands).  I spent some times putting in hacks to get around this, but think I need to tackle the problem from a different angle.

I've been reading neuroscience papers to see if I can get clues from biology.  One interesting paper is Computational Cognitive Neuroscience.  An important clue is this diagram:

The dynamics columns are the key here.  What I have developed so far models only an "integrator" dynamic.  For the reinforcement learning problem, leveraging a "separator" dynamic is necessary.  I've started looking more closely at a thesis paper by Ali Kaan Sungur, who has modeled the basal ganglia and integrated with HTM.

The second issue is that the location signal is kind of a magic box without any description of where it comes from.  For testing, it can be hard-coded, but it needs to be replaced with something generic.  The latest HTM School video introduces the concept of grid cells, which is definitely the correct starting point to solving this problem.  I've also been keeping up with Numenta's work in this area.  Jeff's recent talk at Simons Institute highlights their latest thinking in the current round of research.  The relevant connections:

I've also begun working on a new implementation of HTM in GoLang, which allows for multi-core processing, or distributed processing over a network.  The basic design looks like this:

The basic idea is to integrate a cluster of shards which each process a sub-set of cells or minicolumns in parallel, and assemble their outputs into a completed SDR. This is done by a leveraging a module I'm calling the Sparse Representation Controller (SRC) which takes chunks of a representation and reassembles them:

An SRC acts as a message bus with receivers and transmitters. Shards that need to know complete activation SDRs register as receivers, and related shards register as transmitters to report their output. Once an SRC receives all outputs from transmitting shards, it constructs an activation SDR and transmits it to all registered receivers. Because only the resulting activation SDR is transmitted, the size of traffic within the cluster is small, and most of the processing within a shard can happen in parallel with a relatively small window required for synchronization.

The SRC for Spatial Pooling is a special case, because it needs to coordinate scoring across all shards.  I've called this special SRC the Spatial Pooling Controller (SPC).  The amount of traffic transmitted within a spatial pooling cluster is reduced by having the SP shards only report their top (sparsity * shard count) scores to the SPC. The SPC joins those and clips to just the top (sparsity) minicolumns. It then reports back only those winners which are relevant to each shard for learning.

Other Support / Re: Minecraft Broke in the Most Mysterious of Ways
« on: March 26, 2018, 06:25:58 PM »
The SoundSystem library is unrelated to any graphics issues in Minecraft.  I suggest capturing more of the log, and posting on Minecraft forums.

Do you have a link to a file which experiences the problem?  One thing that comes to mind is possibly some small trailing space at the end of the audio, or the end and start not aligning quite right from a tempo perspective.

Other Support / Re: SoundEngine loop stream sound
« on: February 26, 2018, 03:08:08 PM »
What exactly do you mean? Should the StreamThread add a commend queue to cleanup the source or are you suggesting a completely different way?

Yes, my thought is that all actions performed on a source should be done from the command thread, with callbacks for when various processes are complete, like shutting down, started playing, etc.  This may be the only area that consistently leads to a problem, there are potentially others with the current design (I was quite new to Java when I wrote this, so things could definitely be improved).  On the other hand, if it is working well, maybe not an issue.

Hmm.. that seems like it should work.  Do you mind sending me one of the sound files you are using which has the problem, so I can run some tests?

Do you have a simplified bit of code that has the problem with OpenAL?  My initial guess is that you are using a stereo sound file instead of mono, but that is pure speculation.

As far as missing pitch in JavaSound, that may be due to a control missing on your specific system (not all controls are available on every system)

Yes, in hind sight I should have used callbacks more extensively.  I would suggest that you spin off a new thread for these queries, and have it do a callback when it completes.

Other Support / Re: Sound System problems with cleanup and backgroundMusic
« on: November 27, 2017, 09:30:23 AM »
SoundSystem should only create two new threads.  One thread is for handling streaming sources (it feeds data into the streams, and handles transitions) and the other thread is for processing commands synchronously.  I am aware of a race condition between the command thread and streaming thread, which CreativeMD reported.  Since there is mention of streams in your crash log, it could be related to this defect.

General Discussion / Re: N64 Texture Packs
« on: November 06, 2017, 07:18:35 AM »
No problem, feel free to start a thread (either general discussion or your projects are fine in my opinion).  Just be careful not to post assets that were extracted from actual games (original textures for example).  You can post tools that are freeware or shareware, but avoid posting anything that is copyrighted material that you do not have permission to distribute.

Other Support / Re: How do i stop crashing?
« on: October 09, 2017, 07:11:06 PM »
There seems to be a bit of obfuscation there (class names like "g" and "ave").  Not really helpful as a stack trace without further knowledge.  Perhaps you can go into what you are doing, versions of various software you are using, any modifications, etc?

General Discussion / Re: N64 gaming over the internet
« on: August 30, 2017, 04:42:07 PM »
I'd basically equate it to going over to a friend's house to visit and play games he owns.  Only in this case it is a virtual visit.

Pages: 1 [2] 3 4 ... 174