Using Positive Reinforcement to Assist With Lemur Research: "50 Years of Lemurs at Duke" Exhibit

Were you able to check out the "50 Years of Lemurs at Duke" Exhibit In Duke University Perkins Library at Duke University? As I have mentioned here before, my lemur color vision research was featured in the "current research" exhibit as well as in an interactive video that talks about positive reinforcement. 

Communication & Behavior: Ruffed lemur color vision study

Interactive video: Using Positive Reinforcement to Assist With Lemur Research

If you weren't able to check out the exhibit, don't fret. I'm posting the "positive reinforcement" video here today. It's a short segment on how lemurs are trained using positive reinforcement for lemur research. Watch for my cameo, as well as my very smart black and white ruffed lemur Halley. Enjoy!

I'm presenting my poster at #IPSASP16 tonight

Come hear me talk about training my ruffed lemurs to use SMARTA tonight at the poster session!

Extinction Burst: Throwing Tantrums When There Are No More Free Handouts

Extinction and extinction burst graph from study.com

In psychology, an "extinction" is the gradual decrease of a behavior. When an animal is operantly conditioned (for example, given a food reward to reinforce behavior), reinforced behavior will gradually decrease when that behavior is no longer reinforced with a reward. At the onset of "extinction", an animal will exhibit something call "extinction burst". An extinction burst is when an animal frantically and erratically exhibit reinforced behavior with increased frequency and eventually that behavior decreases and becomes "extinct". I assume it's called "extinction burst" due to a burst of reinforced behavior prior to extinction. Extinction burst is evolutionarily beneficial as animals that exhibit extinction burst is increasing the reinforced behavior for a positive outcome as opposed to those that gives up (extinction without extinction burst). These are important concepts to remember when you train animals. But obviously, these are text book definitions.

Textbook definitions be like ...

I'm writing about this because I am currently experiencing extinction burst with my lemurs. It has been about 4 weeks since I have started training them. Aside from my one star student (Good job, Halley!) all my other lemurs are either starting to associate touching the screen or approaching the apparatus for a food reward. Those that are reinforced to touch the screen for a food reward are currently being shaped (refining their behavior) to touch the red square on the screen for a food reward. However, this requires the lemur to understand that by merely touching the screen will no longer be reinforced and that only touching the red square will a food reward be given.

This new development in cognitive test has given these lemurs much anguish and despair. If you know ruffed lemurs, you know how dramatic they can become. Have you seen those sad face? When their arms and legs are outstretched as if they are saying "FML......". Yea, I've seen it.

Imagine this is a ruffed lemur instead of a sloth

Some of these lemurs are currently exhibiting extinction burst. They are swiping or tapping frantically at the screen while staring at the apparatus chute expecting a food reward. We know that, based on this series of behavior, these lemurs are operantly conditioned by food reward to touch the screen. Shaping them to target the red box instead of any surface of the screen is resulting in some lemurs exhibiting extinction burst. They look like they are throwing tantrums because there are no more free handouts. Compared to previous weeks where they get a food reward for just staring at the apparatus or approaching it.

A black-and-white ruffed lemur. He's just over it.

And to be quite honest, only half of these lemurs are exhibiting extinction burst while the other half are slowly getting shaped to target the red box. Although it is interesting to note that all the lemurs that I am currently training are black-and-white ruffed lemurs. What I would do for these red ruffed lemurs to be exhibiting extinction burst but instead they are herp derping their way into realizing that approaching my apparatus gives them a food reward.

May I Have Your Attention Please?

This is Week 3 at Duke Lemur Center. I had big things planned for this week in my "Things Raymond Wants To Accomplish" list. In reality, it is far more mundane. Not an animal trainer myself (not applied, anyway), but I had thought that training lemurs would be a much quicker endeavor than teaching snails to spell out "YASSS MAMA". Apparently not.

Within the subjects that I am working with, it seems that most of them will take a long time to train. I have lucked out that a pair of lemurs I am working with learned really fast and had gotten a grasp of what I want them to do in my trials. Most of them, not so much. There is a pair of lemurs that I have been training for 2 weeks and they are still learning that the black box inside the enclosure gives them a delicious treat. They still hiss and growl at the box though, and often times will mind their business before returning to the box expecting to find treats.

I wouldn't say I am frustrated because frustration is a useless emotion and useless things are, well, useless. I'll have to say that I sorta expected this and I have already drafted a back up plan before I even fly down here. Back up plan aside, I notice that most of the downtime when I'm training these lemurs is to get them to pay attention.

My initial fear with this experiment was that the lemurs will not be interested in apparatus. However, it isn't so. Each subject's first session with the apparatus was them running towards the box to investigate. Getting them to sustain their attention to the task at hand is the problem. Some of these lemurs would sniff around the box, sit on top of it, run up and around the enclosure before sitting in front of the box for a few seconds (that's the time when I have to positively reinforce them) before moving away again. Training them to associate looking at the box for a food reward seems to be hard enough, let alone reinforcing them that eventually they have to touch the box for a food reward. I find myself oftentimes dispensing food reward just to attract them to come back to the box. But if you want your food reward, you HAVE TO ACTUALLY DO THINGS!

Back to the grind tomorrow ... with hope.

Week 1 at Duke Lemur Center

My training with the lemurs is progressing beautifully. There are already some individuals that have went past the exploration phase and are learning to use the apparatus to get a food reward. Of course, much like the universe, you have the ones that are ahead of the learning curve and ones that are way below. It's important to remember though, that sometimes animals that don't learn certain behaviors or tasks doesn't necessarily means that they are not good learners. An "easy" task or behavior for humans might be hard for animals because it might need a complicated cognitive reasoning or sometimes an animal just doesn't have the attention span to learn something. Although some of my lemurs are still exploring and unsure of the apparatus, I am confident that it is only a matter of time that they'll learn to use it. I just hope it doesn't take months hehe.

Today is Friday and it's the end of my first week at the lemur center. Everything is working smoothly. There were some issues with the apparatus (and no doubt will continue to surface) but I am glad I was able to troubleshoot it ASAP. This weekend is Memorial Day weekend and the lemur center is closed on Monday. I'll have a long weekend and a short training week before I fly back to NYC for the weekend.

Retail therapy. I need a personal shopping assistant and a driver and I'm all set.

Oh I went shopping today. I couldn't resist some retail therapy and I think I deserve it. First, I went to Southpoint Mall (The Streets at Southpoint). I bought a nice pair of Sperry shoes (so comfy!) and some shorts at Brooks Brothers (I love Brooks Brothers). Later, I went to my favorite gourmet grocery store, Southern Season. Cause you know, I'm fancy ;)

All in all, today was a good day.

Rainy Day

It rained today. A much needed respite from this hot weather.

It is wet, cold, and dreary today in Durham, North Carolina. A stark contrast to what it was this week when it was sunny and temperature in the high 80s. Wet, cold, and dreary is definitely my kind of weather. I managed to only sweat a little bit today thanks to the cool weather. Although, since I need to wear latex gloves during training, my hands feel like they are constantly submerged in a pool of water. I got a built in self moisturizing system haha!

I met two new lemurs today. I can definitely see the difference in learning and curiosity to novel objects amongst different age groups. So far, all my animals have been very cooperative and it was very fun training them to use my apparatus. Or maybe they are just training me to give them food rewards.

I seem to have figured out how to fix my food reward dilemma. Stay tuned. Will let y'all know if resolve it tomorrow. One more day till the weekend!

Indoor Training

I moved my training to indoor enclosures today because of the glare issue (I addressed that on yesterday's post). When indoors, the tablet is able to display the stimuli correctly and is more visible than when outside. If you have used your electronics (cellphone, tablet, laptops, etc ...) outdoors when it is sunny and there's glare, you'll know what I am referring to.

I have also made a few changes in the phone app interface but I realize after using it today that it is still not resolved. I had made it show a number when I tell my conveyer belt to "rewind" or "dispense" but it seems like I wasn't able to reset the number back to 0. Instead, if I rewind past the number 0, it stays at 0 when I dispense so that still throws me off with the number of food reward dispensed. I guess I'll have to put in a reset button. Why does it matter to know how many food reward I still have inside the apparatus?

When you train an animal and you give them a food reward, you are not only rewarding the animal but also communicating that the have done a desired behavior. The absence of a food reward during early training sessions will make rewarding and chaining behavior difficult as you always want to reward an animal when they did the appropriate behavior. If not, that will confuse an animal and might make them not associate a desired behavior with food reward. However, because the sound of the conveyer belt is a bridge and trainers can hand the animals a treat, the absence food reward might not be that critical if a trainer offers a food reward right after the sound of a conveyer belt. I might be wrong though ...

I also included a haptic feedback on the phone so whenever I press dispense, I'll feel it. That way, I can focus on the animal while I make my phone app dispense a food reward. That feature worked great and I was able to keep my attention on the lemur without having to look down to make sure I have pressed the "dispense" button.

The craisins I use as food reward have started to stick on the cups because of the heat and humidity (it's sugary). The small grapes that I initially wanted to use are sometimes too big and would get stuck on the cups. Now, the craisins are also getting stuck on the cups. I'm thinking maybe I should use freeze dry fruits but I'm not sure they are as palatable and yummy for the lemurs as these sugary sweet rewards.

Tomorrow, I return and continue training the lemurs indoors. 

This really cute jumping spider decided to check me out as I wait for my shuttle to go back to my hotel. Hey there, lil' guy.

Bridge and Training

A "bridge" in animal training is a stimulus that bridge a correct response to a reinforcement (usually a food reward) and is almost often a vocal communication such as the sound of a clicker or praises (Good boy!). A bridge is usually an anticipation of something positive going to happen and bridging tells an animal that they did the desired behavior and a reward is on its way.

Most of what I did today was bridging and positively reinforcing my lemurs to do the behavior I want them to do. The bridge, in this instance, is the sound of a moving conveyer belt that dispenses a treat. When the lemur hears the mechanical sound of the conveyer belt, it acts as a bridge to tell them they did the right behavior and a food reward is on its way. All the lemurs that I worked with today were very cooperative. They ran right up to the apparatus and did a lot of explorative behaviors but almost immediately knows that the chute in the apparatus drops food reward because they were sitting near the chute (this might be a biased observation on my end). Hopefully, in time, they will associate the sound of the conveyer belt as the bridge.

Aside from doing training today, the trainers and me found a few issues that I need to work on with my apparatus. One of the most important thing was glare. We tried to train the lemurs outdoors and the glare made it impossible to see the tablet screen. I will be training the lemurs indoors tomorrow so hopefully the screen should be brighter (in theory it will). Some design changes were made to the phone app interface, including having a number countdown on how many food rewards were dispensed for easy rewinding, as well as incorporating haptic feedback whenever I press a button so I don't have to look down at the phone while miraculously also keeping an eye on the lemur.

Have I mentioned how unbearable this North Carolina heat is?