Goodbye for now...

    Logan and I returned today to hang out with Despereaux while we worked on our blogs and chatted about our experience with training. Since Des had done such a great job over the last days of training, I brought her a treat- Fruit Loops! At first she was hesitant, but quickly realized how yummy they were and ate them right up!
    Training Despereaux over the last few weeks had its ups and downs. There were definitely some days that I did not want to have to watch her push a bar for 30 minutes and other times where I realized how smart of an animal she really is. Overall, I enjoyed this project a lot. It was neat to see how Despereaux learned over time and I became quit attached to her without even realizing it! I really enjoyed working with a live animal rather than virtual programs because it gave be a better sense of how learning works by using a hands on method. I learned so much from manually shaping Despereaux and watching her learn to press the bar multiple times as we increased the schedules. I also enjoyed having the responsibility to take care of her throughout the weeks of the project.
    One aspect of this project I did not enjoy was Sniffy. I did not learn much from the program in comparison to live rat training. It was time consuming and felt more like busy work than a beneficial assignment. In the future I do not think it is worth the money for students to buy the software and complete the trainings using Sniffy.
    One misconception I had going into the project was that my rat would completely comply to everything I wanted her to do and would always be focused and on task. I quickly realized that she was an animal and would learn at her own pace rather than my own. I became frustrated when she was falling behind the other rats and even did extra training sessions to try to catch her up and increase the schedules as far as I could.
    Overall this was one of my favorite assignments I have had in any class at BSC. I learned a lot and enjoyed playing with Despereaux! I am already looking forward to starting the next project of training Des!

Enjoying her treat! 

Wrapping Things Up

Graphical representation of Despereaux's learning based on the different schedule types. 

Despereaux's target weight (90% body weight) was 205g. 

Extinction

Goal: The goal of these two days was to decrease Despereaux's behavior of pressing the bar until it was eventually extinguished. We wanted to get Des back to her behaviors from the first time she was placed in the box. 

Procedure: We used the "extinction" program that did not reward her for any number of bar presses. Despereaux was placed in the box for 30 minutes without any form of reward or reinforcement. 

Observations and Discussion:

     These were the days I looked forward to the least. I already knew how upset Despereaux would become when she was no longer rewarded for the behavior we worked so hard to get her to perform. Like any other day she immediately went to the bar and began pressing. She would press several times and then check the magazine for her reward and then return to the bar to keep pressing. It was honestly really hard to watch her do this because I felt terrible that she was not receiving any kind of reward and it was difficult to see her frustration. In the first 5 minutes she pressed the bar 76 times. In the next 5 minutes she decreased to 68 presses and continued to decrease to 29 and then 18 presses over 5 minute intervals. For the time period of 20-25 minutes she had 69 presses and 44 in the last 5 minutes of training. There were signs of an extinction burst because she was very spastic and would press the bar 10+ times in a row as if angry. 

Monday (10/7)

     We continued extinction on Tuesday because she was not successful on Monday, but had similar results. She showed an immediate extinction burst in the first 5 minutes with 55 presses. In the 5-10 minute interval she had 0 presses, so we thought she was catching on, but she ended up having a total of 88 presses by the end of the 30 minute session. In between bar presses she would bathe more frequently and sniffed in the left corner a lot. Her presses were also more deliberate and she really emphasized them by pressing very hard.
     I think another day or two of extinction would have completely gotten rid of her bar pressing behavior, but she definitely made improvements from Day 1 to Day 2. 

Tuesday (10/8)

The dark line is Day1 (Monday) and the light line is Day 2 (Tuesday). 

Differences Between Despereaux and Sniffy

     Both Despereaux and Sniffy had several things in common even though one was alive and the other was virtual. Des and Sniffy were both highly capable of learning and responded similarly to the various schedules. Magazine training and shaping were different in each, but the FR schedules were quit similar. Extinction was also a similar experience, although it did not take as long for Sniffy's behavior to become extinct as it did for Despereaux.

    Magazine training was much easier in the Sniffy program than with Despereaux. I do not know if it was just because of the box problems we had in the beginning, but Des took several days of training to even realize the association of the sound/light with food in the magazine.

     I found it much harder to shape Despereaux than Sniffy. Sniffy did not get as distracted or take up so much time with grooming as Des did, and it was much more difficult to keep Despereaux on task. Sniffy was also set to do each step in succession, but with Des she sometimes had to go backwards a little before moving forward.

    From a learning standpoint, I gained the most knowledge from Despereaux. It was much easier for me to see Pryor and Chance's ideas of learning put to use in a live animal than a computer program. It was also more clear that distractions in the room could alter Despereaux's performance, and that could not be seen in Sniffy. I also had a stronger connection to Despereaux and really wanted her to succeed each day of training. These feelings were not the same with Sniffy and I was honestly just glad to have that part of the project done as early as possible.

    I personally do not think I gained much knowledge from Sniffy and would not recommend the program for future classes. It was very time consuming and did not add much of anything to my training of Despereaux. I found the program repetitive and frustrating because it was not as hands on as training a live rat.

Above is a comparison of the FR5 schedule for Sniffy (left) and Des (right). As you can see, both are similar in that they were rewarded at similar intervals over time. 

FR10 Schedule

Goal: The goal was for Despereaux to press the bar ten times to receive one reward. 

Procedure: We used the "FR10" program. 

Observations and Discussion: 

     After yesterday's success I was excited to move on to the FR10 schedule. I knew that this would be the last training day before extinction of the behavior, so I was hoping Despereaux would continue to show progress in her learning. Overall she performed well, with 392 bar presses, but she seemed to be more distracted than she had been in previous days. As seen on the cumulative record, she had a few times where she did not press the bar for several minutes. During these times she would groom, sniff, or just sit in the corner of the box. 

     This FR10 schedule was the furthest we were able to train her with the time constraints. Over the course of the training Despereaux made vast improvements in the number of times she would press the bar. On FR1 she only had an average of 45 bar presses per 30 minute session, and in FR10 she had almost 400! She also decreased grooming and other distracting behaviors as time went on.

Sunday (10/6)

FR7 Schedule

Goal: The goal was for Despereaux to press the bar 7 times to receive 1 reward. 

Procedure: We used the "FR7" program. 

Observations and Discussion:

     Despereaux blew me away today with her responsiveness to the FR7 schedule. She had her most successful day to date and pushed the bar a total of 392 times during the session! Des stayed on task for the entire session and only had a few moments of grooming and sniffing. For the most part she pressed the bar continuously until she heard the buzz of the machine and at that point would dart to the magazine and then return right back to the bar.

Saturday (10/5)

FR5 Schedule

Goal: The goal of this session was to have Despereaux press the bar five times to receive one reward.

Procedure:We used the "FR5" program. 

Observations and Discussion:

     As you can see from the cumulative record, Despereaux did really well with the FR5 schedule! She seems to understand that pressing the bar many times results in a reward and appears to be willing to press it as many times as it takes. She continues to use her head and paws to push the bar. Despereaux also will press the bar several times, check the magazine, and then continue pushing until she hears the buzz of the machine. Her grooming is at a minimum and she rarely sits in the left corner sniffing anymore.

Friday (10/4)

FR3 Schedule

Goal:The goal of this session is to successfully train Despereaux on a FR3 schedule, where she is reinforced for every 3 bar presses.

Procedure: We completed this session using the same software as before, using the program "FR3". 

Observations and Discussion: Despereaux immediately went the the bar when we placed her in the operant box and began pressing. It appears that Des recognizes the bar as the only way to be reinforced. She consistently pressed the bar throughout the 30min session and by the end had 206 bar presses, for a total of 68 reinforcements. She did have some grooming behaviors, but definitely less than in previous days and her sniffing was minimal.

Thursday (10/3)

FR2 Schedule

Goal: Today we moved Despereaux up to an FR2 schedule with the goal being to have her consistently pushing the bar two times to receive her reinforcement pellet.

Procedure: We used the "FR2" program.

Observations and Discussion:

     On Tuesday, we began our session on FR1, but since she immediately began pressing the bar and continued to do so consistently, we removed her from the box after 4 minutes and reloaded the program using FR2. When we placed her back in the box, she returned to the bar and began pressing again. for the first several minutes she would press the bar once and look in the magazine, but after a few minutes she seemed to realize that it takes more than one press to receive the reward. Despereaux had two long breaks where she groomed herself and seemed to lose interest, but after about 4-5 minutes she would return to the bar and begin again. Overall she pressed the bar 37 times and received 18 pellets.

Tuesday (10/1): FR2 Schedule

Tuesday (10/1): First few minutes on FR1

    The next day we continued on the FR2 schedule. Despereaux immediately went to the bar and begin pressing. She pressed 11 times in the first minute and had 27 presses at the 3 minute mark. It was a more successful day and by the end of the 30 minute session Despereaux pressed the bar a total of 163 times. She spent less time grooming today and did not spend as much time sniffing in the corner as past days. Des continues to use both her head and her paws to press the bar, but Logan and I decided this was okay and did not go back to shaping to prevent head pressing. For some reason, the cumulative record for this day is not complete. Although we did a full 30 minute session, only 16 minutes is recorded.

Wednesday (10/2)

FR1 Schedule

Goal: Now that Despereaux is shaped to press the bar to receive her reinforcement, Logan and I aim to establish a consistent Fixed Ratio 1 (FR1) schedule, where she is rewarded automatically for each bar press.


Procedure: We used the "FR1" program and proceeded to place her in the operant box for 30 minutes. The software recorded everything, so we just sat back and observed.


Observations and Discussion:

     Since Despereaux ended yesterday's session on a good note, we were excited to start her on the FR1 schedule. The first 5min were slow and Des was quit distracted. However, by the end of the session she was consistently pressing the bar. It seemed as if she was really learning and improved so much within the 30min session. I was so sad that the 30 minutes passed so quickly and was excited that she was finally showing such large signs of improvement. In this session she pressed the bar a total of 51 times, with the majority of the presses occurring in the last few minutes. I wish the session lasted longer!

Friday (9/27): 

    On Monday (9/30), Despereaux showed great signs of improvement. It appeared as if she was really learning that pressing the bar would give her food and as seen in the cumulative record below she had two periods where she consistently would press the bar. In the first 15 minutes she pressed the bar over 30 times, so we were very hopeful. However, she also still had long periods of grooming and seemed uninterested in the pellets and spent the last half of training in the front left corner of the box sniffing around. We think the regress of behavior may be due to taking the weekend off from training and think she will get back on track after consecutive days of training. 

Monday (9/30)

Corner where Despereaux loves to sniff

Getting Despereaux in "Shape"

     Goal: After spending an additional day making sure Despereaux knew the association between the light/sound and her reinforcement being delivered in the magazine (with the help of Dr. Trench), Logan and I set off to shape her to press the bar to deliver the reinforcement herself.


     Procedure: We used the program "Manual Shape" which allowed us control of dispensing pellets using a hand switch. Before we began we outlined a few small steps that we would reinforce before she ultimately pressed the bar. Keeping Pryor's ideas at heart, we decided to reinforce as follows:

                         1. Des was in the back right corner of the box.

                         2. Des reared up in the back right corner of the box. 

                         3. Des put her nose above the bar.

                         4. Des touched the bar with one paw.

                         5. Des touched the bar with two paws.

                         6. Des pushed the bar.
    We also decided that no matter what, we needed to train Despereaux in the same box every day. It was evident that she became much more distracted by the scents in the other box than her original one and her distraction was compromising her learning.


     Observations and Discussion: Overall, shaping Despereaux took longer than I expected due to operant box problems and her frequently becoming distracted or uninterested. We manually shaped Despereaux for a total of 4 days (30 minutes each day) before we felt confident in placing her on the Fixed Ratio 1 (FR1) schedule. Shaping a live rat was different that what I thought it would be after reading the text. I was not expected Des to be so unfocused and distracted and did not think grooming would be as frequent as it was. It helped to have the steps outlined in advance because in the moment it is hard to distinguish when exactly to reinforce and when to move to the next step.

    On the first day of shaping (9/23), we focused on the first three steps listed above and by the end of the training session Despereaux was doing a fairly good job of placing her nose above the bar. She received 65 reinforcements for her behaviors over a 30min time period and pressed the bar 1 time. It appeared that she was beginning to learn at this point! 

   

Monday (9/23)

     On the second day of shaping, Despereaux was really beginning to catch on. We began the session by reinforcing her each time she placed her nose above the bar. After several minutes she was doing really well with this, so we began shaping her for rearing up over the bar. This was a little harder for her to catch on to, but she eventually did. We ended the session with her placing a paw on the bar without actually pressing it. Although she seemed to do better today, her behavior was still sporadic and she continued to groom quit a lot in between reinforcements. 

Tuesday (9/24)

     Training continued on Wednesday with Logan and I optimistic with Despereaux continuing on to a consistent FR1 schedule. I was beginning to become discouraged because so many other groups were ahead of us and I felt like Des was not learning as quickly as the other rats. However, Despereaux showed some improvements throughout the session. Although she still became distracted by smelling and grooming, she consistently placed her head above the bar and placed her paws on the bar most of the time. She even pressed the bar once!!

Wednesday (9/25)

      On Thursday, Despereaux had her most successful day yet! She recognizes that reinforcements occur when she is near the bar and when she presses it. We still had to do some shaping for the first 10 minutes of the session, but by the end she was pressing the bar. Logan and I got a kick out of her because she tries to do the least amount of work to get her reward and would barely touch the bar with her paw before scurrying to the magazine to look for her pellet. By the end of the session she had 18 bar presses- her largest number to date! I am really optimistic about the rest of training now.

Thursday (9/26)

   

Sniffy- Magazine Training and Shaping

      Before beginning actual training with Despereaux, I completed virtual rat training using the Sniffy Lite software program. I spent time magazine training Sniffy and getting her to form the association between the light/sound in the box and food being delivered into the magazine. Once Sniffy knew where and when the pellet was dispensed I began to shape her to press the bar. To do this I would reinforce her first when she was in the back of the box. I then began to reinforce her rearing up in the back and then when she touched the bar. Next I would only reinforce a bar press and eventually she began to learn that a bar press dispensed a pellet. The program then took over from there and I sat back and watched Sniffy learn! It took about 30min for the program to say she was shaped and on a continuous reinforcement schedule. 

A screen shot of Sniffy in her operant box.

The cumulative record from shaping Sniffy. Each dash mark shows a pellet being dispensed. 

Cumulative record of Sniffy on an FR1 schedule.

     Once Sniffy was completing the continuous reinforcement schedule (FR1, seen above) consistently, I reloaded the program and placed her on FR2. I then proceeded to move on to FR3 and FR5 after adequate time and Sniffy had adjusted to the new schedule- about 20 min. As seen in the cumulative records below, Sniffy quickly adjusted to the increase in work for the same amount of reinforcement. 

A screen shot of the Sniffy workstation.

     By completing the training with Sniffy prior to training Despereaux, I gained more confidence and knowledge of how magazine training and shaping occur. It gave me a better insight to the importance of the first several training sessions and helped in determining the small steps to use while training Despereaux. Using a schedule rather than manually shaping was a lot easier and smoother because Sniffy immediately received the reinforcement without my added reflex time. Sniffy was easily able to adjust and the cumulative records show a consistent pattern of responses. In shaping, Sniffy started out slowly and then increased in responses as she learned, while in a schedule she responded consistently.

     After running several schedules, I placed Sniffy on extinction where she did not receive reinforcement for pressing the bar. The cumulative record below shows how she continued to press the bar, but did not receive a reward. After time, she stopped pressing the bar as often and eventually stopped entirely.

The extinction of bar pressing.

Let the (magazine) training begin!

         Wednesday marked the first day of training for Despereaux! She made it down to her target weight (90% body weight- 205g) and seemed eager to begin learning.

Goal: The goal of these training sessions were to allow Despereaux to acclimate to the new environment of the conditioning box and to become familiar with the magazine where she will receive her rewards (pellets).  Magazine training is a type of classical conditioning where the association between the light and sound in the box corresponds with receiving a pellet in the magazine.

 


Procedure: We placed Despereaux in the box and allowed her time to explore. After a few minutes we gave her one pellet in the magazine using a manual clicker. Once she ate it we gave her another immediately when she pulled her head out of the magazine. We continued to reinforce her with a pellet each time she would pull her head out, but still stay by the magazine.


Observations & Discussion:  When we first placed her in the lab box, she sniffed around and explored ever crevice of the container before heading to the magazine. She immediately began licking the magazine to get all of the leftover chocolate flavored dust from the pellets. She then stuck her entire head in the magazine and continued this behavior several times. Logan and I became concerned with Despereaux's behavior about halfway through the training on Wednesday (Day 1) because she would not pull her head out and kept biting the metal surrounding the magazine. My first thought was that the pellets were not falling into the bottom, but we could see them dropping down through the clear part of the food apparatus. It was obvious that Des was highly frustrated and by the end of the 22 minute training session I was frustrated too because I thought this meant she would be a really hard rat to train and that we were going to have a lot of problems.
        Since her frantic behaviors had increased so dramatically throughout the training, we (Maddie helped!) took apart the box and checked the pellet dispenser. We quickly realized that the pellets were being released and falling, but had been jammed at the top of the magazine. No wonder Despereaux was trying so hard to get her whole body in the magazine- the pellets were stuck at the top and she could smell them!! Relieved, we decided to try the training session again and it went so much better than before!! Despereaux did so well and within 9 minutes had already had 60+ pellets.
      Overall it was a great training day for Despereaux. It was good for Logan and I to experience a problematic behavior from her early in training so we know how to fix it more time efficiently in the future.

Wednesday (9/18): Attempt 2- Des formed a strong association between the sound and food being dispensed. Each tick mark is where she was reinforced with a pellet. 

        On Thursday we returned to train Despereaux and hoped to move on to shaping and eventually establish the FR1 schedule. However, our box was occupied and due to time constraints we had to use a different box. Despereaux did not have a productive day and did not improve during the session.

Thursday (9/19): Despereaux performed well at times, but was too distracted to call it a productive day.

     On Friday we returned hoping for a more productive day. Things were looking hopeful when she immediately went to the magazine; however, Despereaux quickly lost interest and began grooming in the back corner. She then took interest in sniffing the cage and became very distracted, so Logan and I removed the wood chip tray. Des's behavior changed and she just sat in the corner for the rest of the training session. We tried to get her back on track by giving pellets every few minutes, but that did not seem to help. At this point we contacted Dr. Trench and explained our situation and she recommended taking the weekend off from training and starting again on Monday with her guidance.

Friday (9/20): Despereaux was highly distracted and the tick marks seen were mainly to try to get her back on track. 

Becoming More Acquainted

Today I revisited Despereaux during our lab time and spent some more quality time holding her. I already love her and I think she is becoming more comfortable with me! Dr. Trench furthered explained the details of our project and it made me even more excited to begin training next week. Before starting to shape Despereaux, Logan and I have to get Despereaux down to her target weight (90% of her current weight) using slight food deprivation. We begin to train Despereaux next Wednesday and will continue to train daily after that. 

First Impressions

Last Wednesday was my first time to meet Despereaux and I was so excited! I have always loved animals; however, I had never held a rat before and was a little nervous to begin with. In the past I thought of rats as gross and filthy animals, but the rats we are using (Long-Evans) are quite clean and friendly. Logan and I thought long and hard about the perfect name for our furry friend, and decided on Despereaux because it was cute and friendly. I did not hesitate to scoop Despereaux right up to become acquainted with her. Logan, my lab partner, was a little more hesitant and I think Despereaux sensed her nervousness because she pooped on Logan! She is overall well-mannered and did not seem to mind being held for the short period of time. I am excited to get to work with a live rat and look forward to training her in the weeks to come!