Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - Paul

Pages: [1] 2 3 ... 173
Mini Projects / Dad's Song HTM
« on: June 23, 2018, 08:10:57 PM »
Inspired by an interview that Matt recently had with with David Schneider, we have decided to form a group to model in HTM the capabilities required to achieve the zebra finch’s ability to “mimic Dad’s song”.  Once we have accomplished this, we will have covered a lot of the main areas that are needed for embodying HTM, and I will be much closer to my goal of a seed AI for ARTUR.

Anyone interested in joining or just listening in on the verbose interactive brainstormy stuff is welcome to join our Slack channel

Miscellaneous / Re: ARTUR Exmachina
« on: April 24, 2018, 09:30:36 AM »
Also, your project may have an interesting twist if you built an actual insect robot at the same time  ;D

Definitely agree.  While the seed AI would probably not be intelligent enough to do very much interesting with a robot, the hope is that after many iterations through the recursion engine, it could be used to drive a toy robot.

Miscellaneous / Re: ARTUR Exmachina
« on: April 22, 2018, 10:20:06 PM »
The system will need to be built around generic rules, such that certain types of behaviors emerge as a result.  My current thoughts are:

Any given state in the system should be remembered along with the emotional context of that state each time it is encountered.  This emotional context should influence the actions taken in future semantically similar states.  Actions define the possible branches from a given context.  Whatever the particular combination of emotions the creature is experiencing should bias some actions over others, allowing the creature to try a variety of different actions in reoccurrences of a state.

Each emotion would have a range, and would diminish via logarithmic decay.

The level of fear in the creature would be increased based on:
  - Unexpected or wrongly predicted state
  - Level of future punishment predicted by the current state (due to past memories)
  - Actual punishment

When the level of fear is above a certain threshold, the system would narrow potential actions to those which are least flavored by previous fears, preferring those with the least predicted punishment.  This should result in the creature seeking a safe, familiar, predictable state when it is scared (running for cover, for example).  If there are no options remaining, actions which involve the greatest number of motor commands would be chosen (aggressively lashing out, for example)

The level of curiosity in the creature would be increased based on:
  - Unexpected or wrongly predicted state
  - Level of future reward predicted by the current state (due to past memories)
  - Actual reward

When the level of curiosity compared to the level of fear reaches a certain threshold, the system would narrow down potential actions to those which have the least amount of emotional flavoring, preferring those with the highest predicted reward.  This should result in the creature seeking out unfamiliar, new states, allowing it to form a better model of the environment, as well as to seek out rewards.  This should also result in a back-and-forth with fear behaviors (cautiously investigating after a threat has passed, for example)

The level of boredom in the creature would be the inverse of a logarithmic decay (opposite of the other two emotions).  Boredom would increase over time, and be decreased based on:
  - Unexpected or wrongly predicted state

When the level of boredom compared to the level of fear and curiosity reaches a certain threshold, the system would narrow down potential actions to those which have least frequently been tried in the current state.  This should result in a back-and-forth with curiosity behaviors, allowing the creature to seek out new stimuli (when it finds itself "stuck in a rut", for example).  This should also prevent the creature from finding a safe, familiar place, stopping, and never moving from there (since there isn't anything else like hunger or thirst to motivate exploration).

Miscellaneous / Re: ARTUR Exmachina
« on: April 21, 2018, 02:48:53 AM »
Yes, definitely would not need anything like the elaborate range of emotions that humans have.  However, there are a couple which I believe are a basic part of intelligent behaviors:

Fear.  This emotion ranges from being startled to scared to panic.  It motivates the most basic behaviors like capturing attention, freezing, fleeing, and aggression when cornered.

Curiosity.  This emotion ranges based on the novelty of a given context.  Some event first captures a creature's attention, and any threat has either passed or is below some threshold.  The creature is motivated into an investigatory behavior to accumulate more information about the event.

Boredom.  This emotion ranges from malaise to frustration to exasperation.  While this one is a more complex emotion than an insect probably has, I think it is necessary in a non-biological intelligence to replace other drives like thirst/hunger/sex which motivate exploratory behaviors.

Miscellaneous / Re: ARTUR Exmachina
« on: April 20, 2018, 04:17:08 PM »
Yes, that is exactly what I am thinking.  It is interesting that the path to intelligent robots is could end up being backwards from how it is imagined in Sci-fi (where emotions tend to be absent or developed much later)

Miscellaneous / Re: ARTUR Exmachina
« on: April 20, 2018, 03:08:13 PM »
The significations of that paper here, I think, is taking it in context of the amygdala:

The relating that to Figure 2 from the paper:

The amygdala is outputting to the prefrontal cortex (PF), which is a hub in the distributed circuit for semantics, which are grounded in sensory motor activation.  This means the sensory input, and all semantic representations derived from it, and generated motor outputs, are all flavored by emotional context.  The obvious conclusion is that emotional context is a critical part of the sensory-motor circuit and action decisions.

I interpret this to mean that in order to understand concepts like "curiosity" that are important for more efficient reinforcement learning strategies, the above circuit (with its emotional input) need to be understood and part of the model.

Miscellaneous / Re: ARTUR Exmachina
« on: April 20, 2018, 09:29:14 AM »
I'm targeting more of an insect-level intelligence (which I believe is achievable with current technology).  But you correct that curiosity is an absolutely essential element for any autonomous agent.  The agent must have some core motivation to try something different in a recognized context (or try anything at all in a novel context).

Deep Mind (and similar RL implementations) solve this with a diminishing random chance of non-ideal behavior in any given context.  This definitely works, but it has its own problems.  Training requires huge numbers simulations where the agent essentially brute-forces the problem until it reaches a policy that is deemed to be acceptable.  As we recently saw with the Uber self driving car accident, this may not be good enough in situations where human safety is a concern.

My current thinking is that the key to understanding curiosity (which is critical to effectively implement reinforcement learning) requires first understanding emotions.  I recently read an interesting paper by Friedemann Pulvermuller which goes into the relevant biology in nice detail with excellent visualizations.  It appears that emotional flavoring of sensory motor mechanisms is at the core of how the brain establishes a semantic grounding (and thus is a critical part of choosing what actions to take in a given context).

Theory and Discussion / Re: What is ARTUR?
« on: April 20, 2018, 08:29:01 AM »
I've been quite here on the forum for a while (mainly been active on the Numenta forum).  Thought I would post a progress update.

I spent some time on the Reinforcement Learning problem, and linking it with HTM (which still I believe is the best starting technology for the seed AI).  About this time last year I formalized a strategy, which I described in this thread .  For reference, here is a visualization of that strategy:

I've since discovered a couple of problems with this strategy.

One issue is with the reinforcement TP layer.  The basic problem here is that although the pooling layer creates a representation depicting all expected future rewards/punishments, the result is that the system picks a future motor command with the most reward (skipping required previous motor commands).  I spent some times putting in hacks to get around this, but think I need to tackle the problem from a different angle.

I've been reading neuroscience papers to see if I can get clues from biology.  One interesting paper is Computational Cognitive Neuroscience.  An important clue is this diagram:

The dynamics columns are the key here.  What I have developed so far models only an "integrator" dynamic.  For the reinforcement learning problem, leveraging a "separator" dynamic is necessary.  I've started looking more closely at a thesis paper by Ali Kaan Sungur, who has modeled the basal ganglia and integrated with HTM.

The second issue is that the location signal is kind of a magic box without any description of where it comes from.  For testing, it can be hard-coded, but it needs to be replaced with something generic.  The latest HTM School video introduces the concept of grid cells, which is definitely the correct starting point to solving this problem.  I've also been keeping up with Numenta's work in this area.  Jeff's recent talk at Simons Institute highlights their latest thinking in the current round of research.  The relevant connections:

I've also begun working on a new implementation of HTM in GoLang, which allows for multi-core processing, or distributed processing over a network.  The basic design looks like this:

The basic idea is to integrate a cluster of shards which each process a sub-set of cells or minicolumns in parallel, and assemble their outputs into a completed SDR. This is done by a leveraging a module I'm calling the Sparse Representation Controller (SRC) which takes chunks of a representation and reassembles them:

An SRC acts as a message bus with receivers and transmitters. Shards that need to know complete activation SDRs register as receivers, and related shards register as transmitters to report their output. Once an SRC receives all outputs from transmitting shards, it constructs an activation SDR and transmits it to all registered receivers. Because only the resulting activation SDR is transmitted, the size of traffic within the cluster is small, and most of the processing within a shard can happen in parallel with a relatively small window required for synchronization.

The SRC for Spatial Pooling is a special case, because it needs to coordinate scoring across all shards.  I've called this special SRC the Spatial Pooling Controller (SPC).  The amount of traffic transmitted within a spatial pooling cluster is reduced by having the SP shards only report their top (sparsity * shard count) scores to the SPC. The SPC joins those and clips to just the top (sparsity) minicolumns. It then reports back only those winners which are relevant to each shard for learning.

Other Support / Re: Minecraft Broke in the Most Mysterious of Ways
« on: March 26, 2018, 06:25:58 PM »
The SoundSystem library is unrelated to any graphics issues in Minecraft.  I suggest capturing more of the log, and posting on Minecraft forums.

Do you have a link to a file which experiences the problem?  One thing that comes to mind is possibly some small trailing space at the end of the audio, or the end and start not aligning quite right from a tempo perspective.

Other Support / Re: SoundEngine loop stream sound
« on: February 26, 2018, 03:08:08 PM »
What exactly do you mean? Should the StreamThread add a commend queue to cleanup the source or are you suggesting a completely different way?

Yes, my thought is that all actions performed on a source should be done from the command thread, with callbacks for when various processes are complete, like shutting down, started playing, etc.  This may be the only area that consistently leads to a problem, there are potentially others with the current design (I was quite new to Java when I wrote this, so things could definitely be improved).  On the other hand, if it is working well, maybe not an issue.

Hmm.. that seems like it should work.  Do you mind sending me one of the sound files you are using which has the problem, so I can run some tests?

Do you have a simplified bit of code that has the problem with OpenAL?  My initial guess is that you are using a stereo sound file instead of mono, but that is pure speculation.

As far as missing pitch in JavaSound, that may be due to a control missing on your specific system (not all controls are available on every system)

Yes, in hind sight I should have used callbacks more extensively.  I would suggest that you spin off a new thread for these queries, and have it do a callback when it completes.

Other Support / Re: Sound System problems with cleanup and backgroundMusic
« on: November 27, 2017, 09:30:23 AM »
SoundSystem should only create two new threads.  One thread is for handling streaming sources (it feeds data into the streams, and handles transitions) and the other thread is for processing commands synchronously.  I am aware of a race condition between the command thread and streaming thread, which CreativeMD reported.  Since there is mention of streams in your crash log, it could be related to this defect.

Pages: [1] 2 3 ... 173