Social Neuroeconomics – Over bidding         

Raphael Spiro QC 12 and Jim Stellar

Raph has been reading a bit of the literature on neuroecomomics, something we have written about before in this blog.  We’d like to propose that the same brain areas that are involved in making decisions about purchases made with money are also be involved in “purchasing” with time and effort one’s major or field of study in College.  In particular, we wondered about the influence of others on these major-choice decisions.  Raph came across a series of studies that deal with the influence of others on decisions such as an auction where one is bidding against competitors.  Let’s begin by asking Raph to tell us about one of those studies.

Most people put a lot of thought into their economic decisions, and while we’d all like to think we’re above the influences of peer pressure and social competitiveness, it would seem that no amount of planning or intelligence can fight our natural inclinations.  In a 2008 study by Delgado et. al, brain imaging technology shed some light on the way in which the people around us affect our money management plans.  Participants in this study took part in two potential money-making tasks.  In both tasks, participants were asked to place a bid for the round.  If their bid was closer to the target value for the round than the competitor’s bid, they won the round and kept the difference between their bid and the value.  If they overbid, they did not win anything, nor did they lose anything.  The first task was an auction in which participants were pitted against a human competitor.  Before the auction, the participants met their counterparts.  In the second task, participants played a lottery, where the outcome was determined by a computer.  Unbeknownst to the participants, the computer and the human competitors used the same strategy when betting, based on a Nash Equilibrium.

The study found that when participants lost against their human competition in the auction game, they experienced a decrease in activity from baseline in their right ventral striatum (or what we also call the nucleus accumbens, a brain reward area).  This activation pattern is commonly seen in situations when money is lost.  Additionally, participants who experienced greater decreases in activity in their right accumbens after a loss were more likely to overbid.  That is to say, the participants who felt the effects of losing in front of a peer more were more likely to overbid.

Fascinating that the effects of losing in front of a peer was related so strongly related to overbidding, especially since we know from the work of Brian Knudson at Stanford  (and others) that believing one is going to make money activates the accumbens. 

One explanation for the results stems from the possibility that the participants in the study were not responding to a loss of money or points.  The loss of an auction carried no penalty.  Generally, a situation such as this would not be expected to produce decreased activation in the right accumbens.  The researchers suggested that the activation pattern produced was a result of the perceived loss in front of a peer.  This “loss reaction” and the overbidding it produces can be related to learning in and out of the classroom.  Both school and work place are rife with social competition and possibly analogous situations to the one examined in the Delgado et al. study.  In school, receiving marks on a test that are lower than the class average or than a specific classmate’s remarks might elicit a similar “loss reaction.”  In the workplace, receiving a bad review or being passed over for a project opportunity could produce similar effects as well.  While overbidding may be slightly maladaptive in the case of a monetary auction, in a work or school setting it seems like a positive reaction.

Overbidding in these settings would ensure a more successful outcome in all endeavors.  “Overbidding” on a test translates to over-studying.  A student who feels a very strong effect of losing will be likely to overinvest their resources (in this case time and effort) to ensure that they succeed on their tests.  Similarly, an employee or intern will work hard to ensure that they are the best candidate for a promotion or special project.

Social cues may can influence these decisions and lead to herd type behavior where one commits too much to an object (or a major?) on the advice and behavior of others.  This was shown in another fMRI accumbens study by Burke et al. 2010, where subjects were given the chance to buy stocks after observing the buying decisions of others (a group of 4 faces marked with a check for having bought the stock).  If all 4 faces were marked as having bought the stock, or did not buy the stock, the % buying by the subjects swung from 80% to 20% - the herd effect. The researchers found the subjects were also influenced by the pictures of chimpanzee faces and by data on the stocks themselves, but not as much as the herd.  The accumbens reacted to the herd effect and the authors argued that it did so independently of, and perhaps driving, the buy decisions.  Other brain areas (amygdala, anterior cingulate cortex) were involved, particularly when the buyers went against the herd, but that is another topic.

What does all this mean for experiential education?  For us it gives new level of power and weight to one’s reflections on the gut decisions in deciding if that experience is right for us.  For example, in a prior job, one of us (JS) saw that students studied abroad independently engaged in much less reflection than students who went as a group led by a faculty member.  We thought these students, participating in a program in Cairo, were sitting in their rooms at night talking over the day’s events and reflecting on lessons they had learned as they adapted to a culture and perspective that differed from what they had experience in America.  In that environment, some might have decided to become international human rights lawyers/workers.  If others in the group were also making such decisions, it is not hard to see how one could be sucked in.  In addition, this group decision could have been perceived similarly to bidding (if it took on a competitive air), leading to an occurrence of the over-bidding phenomenon, as one did not want to be perceived as making the wrong decision.  This is a common group effect that we have seen many times in social psychology and in the general course of human affairs.  But to us, it takes on a certain power and precision being linked to brain areas and quantitative economic studies.  The question is how do we in higher education leverage these phenomenon to produce better engagement and learning by our students in and out of the classroom?

• 1 Comment

We experience so that we may learn.

Jungyo Kim QC’13 and Jim Stellar

From my perspective (JS) the story begins when JK took a class I co-taught last spring (PSY282) and we connected over an idea for a new club she is starting to promote emotional, mental, and physical health, on the Queens College campus.  But that led to conversations about learning from experience and that led to her joining a small group I run that explores the intersection of experiential learning in higher education and what might be called social neuroscience.  As part of that group, we had a conversation recently about how passion for a field or a project develops in a college setting.  We agree that passionately committed students do better in college as they put more time into their studies and may even just think more deeply about the subject matter in and out of class because they are so committed.  But the question is how one develops that passion in a student. 

From my perspective (JK), the story begins with the day back in last spring, when I first approached JS after a PSY282 lecture on eating behaviors and disorders. I had a growing inclination to talk to JS as the lecture came to an end; the information from the lecture struck me right through the heart. I had overcome an eating disorder myself, and it had always been my interest to raise awareness and help with this issue. After some hesitation on whether approaching him and his co-teacher was even a sane idea or not (after all, I was about to tell my professor that I was a recovered anorexic and that isn’t something you just yell out simply because you’re bored), I had made up my mind to take this on. And it’s interestingly one of best things I’ve ever done.

So, let’s talk about experience.  What do you think that teaches you as a college student and as a person?

I think one of the most beautiful things about life is that we as human beings have the stunning ability to take from all of our experiences from every moment of every day and learn, no matter how small, big, joyful, or painful these events may be to us. It’s just a matter of being aware of those events and their effect on us, and choosing to extract the good from them. I’m a firm believer in the idea that experiences are crucial to our growth, happiness, and health. My personal struggle with anorexia nervosa and its aftermaths was undoubtedly a painful experience, but one of the most rewarding events of my life that I am very thankful for.  

I’m grateful that I have learned. I no longer take myself or my health for granted. I truly appreciate even the silliest sounding things (like the existence of organic rose oil fragrance- it is heavenly) to the greater things we would all agree upon (like family). I’ve learned what it means to be truly happy. I am the healthiest, and happiest I’ve ever been. And I continue to grow.

What is learning to you?

Learning is a gift and a privilege. We start learning from the moment we take in our first breath of air. We also learn in school- as we try to write down our ABCs on a sheet of paper that seems too small to hold all that we want to scribble. We grow up to decipher the logic of chi squares statistics in college and the reasons to why the human brain feels pleasure when seeing a piece of chocolate cake. But we should never forget that we learn some of our most memorable, treasurable lessons through our personal experiences- whether it be the argument you had with your boyfriend over the true meaning of a love song, whether it be the first time you realized how thankful you are for your mother and all of her silly scolding, whether it be the old friendships you had chosen to end when you saw that the pain you were giving each other was too much too soon, whether it be the eating disorder that taught you to do all that you can to not have another soul go through the pain you went through.

Your story reminds me of a book I read by Chopra on leadership and the soul and another by Schwarz and Sharpe on the origins of practical wisdom.  It will not surprise you that these books, to me, emphasize learning from experience.

It is my plan and goal to apply my knowledge from experience to the process of leading an organization. My idea of a good leader has, among other traits, the willingness to understand the point of views of the people around her, whether those views agree or disagree with his/her own. Compassion and the ability to understand others are qualities that I felt lacked in eating disorder treatment centers of my personal experience. When I do start my organization, one of my goals would be to promote a clearer understanding of the “how” and “why” behind such important issues as eating disorders. I think one of our greatest strengths as human beings, is our ability to feel what other people feel; we are able to understand others if we try.

What other ideas do you have for your organization, and how do they relate to experiential learning?

I plan to connect the many different interests of studies on the QC campus such as psychology, history, literature, and art, with the importance of total health. Making connections is crucial to learning. I want this organization to sparkle and ooze experiential learning from within- with members taking part in fun, memorable activities and events that spark thinking, questioning, understanding, appreciation, and awareness. Student organizations are one way of strengthening what the students learn in class, all in an enjoyable and a refreshing way. In the long run, experiential learning as such would give students invaluable qualities and skills to be cherished for life.

So, in conclusion, why are you passionate about experiential learning? Why is it important to you? 

Learning is a big part of my happiness. Learning is a big part of my life. Someone wise once told me: “Give the world the best you have, and it may never be enough. Give the world the best you’ve got anyway.” Sounds ludicrous to do something just for the heck of it- with nothing material to gain, doesn’t it? But I give the world the best I’ve got anyway, simply because I want to, simply because I know it makes me happy, simply because it gives me strength. I will continue to give the world my best by being true to all that is around me, and by taking what I have learned, learn, and will learn, to help those around me. I experience so that I may learn and as the Queens College motto says, “I learn so that I may serve.”

This conversation between us, illustrates where passion in learning comes from.  It seems to us both that it comes from personal experience.  In this case, it started with JK’s health condition, refracted through a class in which professors and student connected as people, and then went to something JK is building on her own (the club) outside of any curriculum. This real-world (“every day”) aspect of the experience is what gives it sustaining power when coupled with the student being in charge, when it is personal, and when it matters.  By contrast, a classroom can be a place where the student is not in charge, just watches a very nice display of knowledge, and is left in only a few minutes of question in or out of class to touch the material personally.  But in a service-learning project or even in an internship or abroad experience, the student is engaged because they cannot just watch.  They must participate and, in many cases, even lead and thus take that leadership responsibility.  These traits engage the Hidden Brain (another book we suggest you read), pump up the limbic/motivational system, and otherwise stimulate what we are calling the “other lobe of the brain.”  Then with a connection to the content area (after all this is college), we have the most effective higher education possible because we have a passionate student learner.

• 4 Comments

Refining a long term passion in the academy

Michaela Tralli QC’12 and Jim Stellar

Michaela took a course I co-taught last spring.  We began to talk about how one finds one’s field and her field looks like it will be direct patient care in clinical psychology.  Often in this blog we write about how one discovers one’s passion through a combination of course work, internships (or some other form of experiential education), and mentoring.  But what about the people who came to college with a pretty good idea of what they want to do?  Where do those ideas develop and how does “other lobe of the brain” thinking apply to those cases?  Michaela may well be one, so let’s get her to tell us her history from the perspective of her long-term interest.

My interest in clinical psychology first developed when I was a young seventh grader.  To understand why my passion developed this young, it is important to know about my earlier education.  Growing up, I was not a great student.  I never got higher than mid-70’s except for a rare 80 here and there.  It was the same when I started middle school in seventh grade.  In most of my classes my grades were the usual.  I received mostly 70’s.  Maybe an 80 if I was lucky.  However, when I started health class, I grew so interested in everything we spoke about.  Whether it was drugs, physical or sexual abuse, family issues, or nutrition- I loved it all.  My passion for this information stuck out right away.  I studied hard for the class, not because I wanted a good grade (clearly that was not important to me), but because I LOVED what I was learning about.  The end of the semester came and I received a 100 for the class.  This was my first 100 that I remember receiving.  When I told my parents, they were ecstatic.  They praised me and told me how proud of me they were.  I felt great.  The love I had for this information, and the praise I received made me realize that I wanted to help people who were dealing with some of these issues, whether it was drug addiction, obesity, family life, and so forth. 

This story is terrific and I see myself in it.  So, let’s take the next step and talk about what happened when you got to college and how you ended up eventually in Psychology with a focus on being a one-on-one clinical practitioner.

When I first got to college, all I knew was that I wanted to help.  I started my education at Nassau Community College as a liberal arts major, since I was unsure of what field I wanted to work in.  Luckily, this major required that I take many credits worth of electives.  I took full advantage.  My courses ranged from nutrition to zoology, and everything in between.  For a while I thought about becoming a veterinarian so that I could even help animals.  However, by the time I graduated from Nassau, most of my credits were gained in nutrition classes, all of which I got straight A’s in.  This feeling of success was familiar.  I thought that nutrition was my calling.  I came into Queens College as a Dietetics major.  I loved most classes, and passed many with A’s.  It was the four credit sciences that gave me great trouble.  It took me two tries to pass basic chemistry, and when organic chemistry rolled around, that familiar feeling of failure hit.  I decided to seek the help of an advisor here at QC.  With her guidance, I realized that if I really wanted to be a nutritionist, I would find a way to pass organic chemistry.  After some thought, I discovered that I wanted something with more substance- something that I would be proud of, and that would enable me to encourage people who were suffering mentally to not give up on achieving the greatest life possible.  My advisor first suggested social work, and then she said- “clinical psychology.”  Clinical Psychology.  That was it. I questioned myself:  “How did I not think of this before?”

So it was an advisor who made the connection that clicked in your head, not an experience on an internship or something you did outside the academy.  We always say that internships and other forms of experiential education are a great supplement to a fine classical course-based education.  Your experience reminds us of the power of the classical education.  But, let’s pursue it a tad further.  Since that moment, you have obviously confirmed in your mind that this is the right course.  How did that happen…further course work, anything you experienced in the clinical psychology field?

It happened for me through personal experience with a social worker and a psychiatrist.  Over the past year, I had to begin therapy due to chronic severe panic attacks.   After a few months of seeing my therapist, I began to understand why these panic attacks occur.  I was getting better in that I understood the source of the pain, but I was still suffering tremendously.  Realizing this, my therapist worked together with a psychiatrist to get me on the right medication.  Understandably, I grew extremely close to these two women.  I started to realize how much they love their jobs.  They are passionate in what they do and they truly want to help.  My therapist even insists most weeks that she does not want to take my copay because she wants as little stress as possible in my life.  Seeing how happy they are when they help made me further realize that I want to be just as happy in my career.  It was then that I confirmed that this is the right course for me. 

Before we go into some analysis of what we just wrote (typical for this blog), I want to thank you for sharing your story with us.  It is compelling.  I understand now, for the first time, from where your self-confidence comes in your career choice.

What can we extract as a lesson from this conversation?  First, it is the power of teachers, there at the right moment in the classroom to spark an interest and change a life.  We must always remember that experiential education is the supplement to the facts and theories supplied by a classical education; and that this knowledge is given by people who have the power to touch us in what we might call the “other lobe of the brain.”  Next, it is the power of reinforcement.  The teachers who provide us with knowledge are, in fact, responsible for sparking an interest.  But without the reinforcement of others, that interest may never be pursued, and therefore, may never become a passion.  There is more, and it comes in this story through two therapists who modeled the behavior of loving one’s career that, like other forms of experiential education was transmitted personally, often without intention, seemingly as a side effect, but becoming a main effect, powerfully affecting the career.  Teaching by example, and reinforcing one’s success is just a piece to a much larger puzzle in expressing what is happening in experiential learning.

• 0 Comments

Sensemaking – something one learns from experience

Luisa Melo NU’00 and Jim Stellar

Luisa was an undergraduate who worked on a senior honors project with me in behavioral neuroscience.  After graduation she worked in academic biochemistry research for years, pursued a Master’s in International Relations at the Fletcher School and has wound up in the Bentley University PhD program in Business. It was there she came across a concept in one of her business classes that begins our conversation and is taken from one of her papers.

Karl Weick, a professor at the Ross School of Business at the University of Michigan, developed the action perspective in Organization Studies. It is relevant to experiential education and came out of his interest with cognitive dissonance.  The idea is that the human mind requires consistency when aware of conflicting results, and is flexible and resourceful in obtaining that consistency (Kogut & Zander, 1996; Weick Sensemaking in Organizations, 1995). Accommodating his psychological work to the business organization, Weick noted that “organizations begin to materialize when rationales for commitment become articulated.” He defined sensemaking as “committed interpretation” and as the “infrastructure of organizational inertia” (1977). Sensemaking in his view, the ability to accommodate cognitive dissonance, kept organizations from changing and highlighted a ‘reality’ of organizational life.

I would add that sensemaking sounds a bit like someone I have cited before, Barry Schwarz, who talked on TED about practical wisdom and with Kenneth Sharpe wrote a book in 2010 about the subject.  Practical Wisdom seems a bit like sensmaking.

Weick (1995) outlined the process of sensemaking as constituted by enactment, justification, selection and retention. Enactment is the action part of sensemaking, which by definition requires committed action. Weick pointed out that managers (or actors we believe to be important in an organization) acted first, then evaluated their action as to whether it was socially acceptable. Enactment (Weick 1977) “brackets raw data” and generates it, and is made up of four components: self-fulfilling prophecies, commitment, social information processing and retrospective sensemaking. Self-fulfilling prophecies capture the idea that people interpret things that happen as inevitable but actually bring them about through their actions. Commitment involves choice, irreversibility and public awareness of a decision. Weick quotes Pfeffer & Salancik who described commitment as “binding individuals to their behavior” (1978). Public awareness leads to evaluation and justification of an action socially acceptable. Retrospective sensemaking refers to the idea that action happens first, then is interpreted as having happened for a reason. Selection and Retention complete the cycle of sensemaking, and refer to the idea that interpretations of action are infinite. Actors ‘select’ those interpretations that they can accommodate, and retain via codification so that the “meaning of enactment is preserved in organizational memory.”

OK, enough with the professional speak. You went to Northeastern University, a co-op school, and had plenty of work experiences.  A lot of us regard you as having a high level of wisdom even as a college student that perhaps you gained from your experiences immigrating to the United States from Colombia as a child.  But to focus, what did you learn that you could call sensemaking from your experiences, particularly back in the day when you were an undergraduate.

Well, this is a really tough question! I would say there are a few ways to answer it. On one level, Weick’s greater point was that our chosen organizations are entities that do not function in the way we think. In a way, we create them based on our enactment. You can argue that a university education prepares us (i.e. socializes) for professional life and that suggests we enact based on whatever we have learned. However, the reality is that sometimes our education does not prepare us for how we behave as professional or how we have to adjust our behavior.

The opportunity for experiential education can challenge this expectation of rationality, or one-to-one correspondence very early on. My first co-op experience was one reason why I changed to behavioral neuroscience. I chose biology as the standard way to achieve a pre-med degree. My experience working in a community health center in pediatrics showed me that maybe there was a connection to be made between mental health and general health.

Another concept that Weick explores is that of improvisation. It comes out of Claude Levi-Strauss’ work on bricolage. Weick saw all of us a bricoleurs, or jacks-of-all-trade. He argued that working on your craft helped to build your tool box, and that the more fine-tuned this box was, the greater the ability to improvise. There is a tendency to use the concept of improvisation as building something out of nothing, but Weick and the jazz greats really improvise based on ‘something’. In that sense, sensemaking requires improvisation and experiential education is one way that we keep fine-tuning our tools.

When I worked in academic research after graduation, there were problems I faced that I instinctively improvised to solve. A coworker who had done exceptionally well at another Boston-area school once asked me how I was able to address so many issues. Who, she asked, had taught me, and why wasn’t she being taught. At the time I said I was “making it up as I went along because you have to solve these problems, you have to be creative”. In retrospect, I think my experiences working had fine-tuned the toolbox I took for granted at the time.

Way back in this blog’s history, is a post calling for industry and academics to partner to “co-create” the next generation of educated citizens that are work ready, who can sensemake out of their experiences at the moment and get to a goal when there is no path.  Classical Higher Education today is said to be out of step with what is needed in the workplace generating graduates who cannot sensemake (our words). We believe that this is situation can be fixed if colleges and universities would not only educate the cognitive lobes of the brain with facts and theories that establish knowledge and thinking ability, but also educate the “other lobe of the brain” which is more instinctive, cunning, sensemaking.  This is done with unstructured experience in a real world setting where what the student does matters to someone whether it is getting an experimental protocol right in a lab as an undergraduate research assistant or working in a business where if money is not made, people do not get paid.  Internships, service-learning, undergraduate research, and even abroad experiences in different cultures all do that.  Then, even in a struggling economy, a job awaits … but so does graduate, law, medical, or even business school.

Our references:

·         Daft & Weick, (1984), Toward a Model of organizations as interpretation systems. The Academy of Management Review, 9:284-95. 

·         Kogut & Zander (1996)  What firms do? Coordination, Identity, Learning. Organization Science, 7:502-18.

·         Weick, Karl. (1995). Sensemaking in organizations. Thousand Oaks, CA: Sage.

·         Weick, K. (2001) Making sense of the organization, Blackwell Pubs. This is a collection of Weick’s works, organized by him, with introductions for each section. From this collection, I cite:  “Sensemaking in organizations”, 1993; “Sources of order in underorganized systems”, 1985; “Organizational redesign as improv”, 1993; “Enactment Processes in organizations”, 1977

• 0 Comments

The Undergraduate Experiential Education of an MBA

Corinne Freeman D’Ambrosio NU 02 and Jim Stellar

Corinne was a major in Anthropology with an Education minor who went into business after graduation and just received her MBA degree (congratulations!).  She also was head of the College of Arts and Sciences Student Advisory Council when I was Dean of the College at Northeastern and thus had a serious undergraduate leadership position.  We agreed to write something here about what a deep co-op based experiential education undergraduate education had to do with leadership and business entrepreneurship almost 10 years later.

So, here is my first question.  What do you remember from your co-op/leadership undergraduate experience that best applied to the entrepreneurship you lived in your years in the business world?

Despite the seemingly large divide between my undergraduate co-op positions related to Anthropology and my current occupation in technical sales, there were many valuable lessons taken from those early experiences.  Most important of these lessons was a determination to push through seemingly insurmountable obstacles in order to reach goals, both large and small.  I remember changing diapers at East End House in Cambridge as part of my very first co-op at Northeastern.  This was when I was developing an interest in Education and wanted to see how well I did with children.  Not well, it seemed, with 2 year olds!  Part of the co-op experience is learning what you DON’T want to do as much as it is learning what you excel at.  For many graduates of traditional 4 year universities, after diligently completing undergraduate work, upon entering the workforce in the chosen field the new graduate is faced with the shock, “I hate this job!” 

Many people say that on average, each person changes careers 5-7 times over the course of a lifetime.  Not just jobs (which most people will hold 7-10 different ones) but CAREERS!  I would like to think that my co-op experiences at NU helped me to eliminate one or two of those career changes along the way.  While I am no longer changing diapers, I believe the co-op experience helped to give me a stronger sense of self, a sense of self that gave me the confidence to go back to business school ten years later because I KNEW what goal I wanted to reach.  Those early experiences helped to bridge that continental divide between education and application, and served me well during completion of my Master’s program.

Some say that the students that do not do co-op or other serious internships and pursue a classical “ivory tower” academic education simply catch up in the first few years of work. Think about your first few years of work and comment on what if anything is enduring from experiential education?

The work ethic derived from full-time professional experience as an undergraduate is not something a student can “catch up on” if their choice has been to pursue a traditional “ivory tower” rather than co-op based education.  The reasons I have for making this statement is two-fold.  First and foremost, doors will open easier for those who graduate with an existing resume beyond flipping burgers or working in the local ice-cream parlor part-time.  Not only does co-operative education provide the classroom learning needed to perform a particular job, but real proof that a job candidate can apply that learning in the workplace setting.  Not only are the references from past co-op positions invaluable when trying to secure a first job out of college, but the networking gleaned from making and maintaining all of those contacts through the years multiplies a job applicants chances.

Secondly, there is a huge paradigm shift going from student to employee.  After years of full-time co-op positions interspersed with full-time study, the co-op program graduate has less of an adjustment to make, and in my opinion, will be able to hit the ground running more than a candidate graduating from a traditional program.  Sure, both will learn on the job, make contacts, and have to adjust to “real life”, but the co-op graduate has an advantage in all of these arenas. 

Why is this?  From where does this advantage come?

I would like to give two examples of why I feel strongly that my opinions given above are based on fact and not simply prejudicial judgment because of my own choice in education at Northeastern, a school with a longstanding history steeped in cooperative education. A good friend of mine from my graduating class took a co-op job working for the student radio station.  After gaining a following on campus, he was offered the opportunity to work as an intern for local radio station, KROQ.  After proving himself once again, he was offered a paid co-op position at KROQ, which led to his immediate hire after graduation at MTV in New York.  Without the contacts and experience from that first co-op at the school station, none of this would have been possible so early on in his career. 

Another less glamorous example compares my career track to that of a good friend of mine who graduated from a traditional program at UMass.  After working at East End House and discovering I was not cut out to work with toddlers I thought I would try my hand with school-aged kids and teenagers at the Wang YMCA of Chinatown in Boston.  After working as a summer camp counselor for two years, I was hired as camp director the next.  By the time I graduated from Northeastern I was immediately offered a full-time position as Youth & Teen Program Director with the YMCA, which I loved every minute of until I moved to California to pursue other things.  My friend who graduated from UMass was still working her way through the ranks at another YMCA since she had to start from ground zero as a counselor in their before and after school program.  She was hired upon my recommendation to fill my position after I left, and has since gone on to do wonderful things, including attaining her Master’s degree.  I believe that as such a talented and motivated young woman that she would have been years ahead in her career if she had the same opportunities I had through the co-op program. 

Notice above the clear practical advantages that Northeastern and other Cooperative Education schools often tout – a job after because along the way the student built a credible record in the eyes of the employer.  But there is something else we feel that may be more important and that is the development of the student.  Turn those eyes around from looking to the career and look at the student in college, going through the program, growing, finding out what they want to do, developing a passion for it, learning to negotiate a complex world beyond the structure of college, and getting a kind of confidence that matches the enhanced resume we just discussed. 

In a world of charges that Higher Education is not delivering (see “Academically Adrift”) and in a flood of new books that tell us that motivation is most properly intrinsic in a complex modern world (see “Drive”) the most important piece to really getting somewhere is to fire that internal passion to put in the time to develop into something awesome (see “Talent Code” or “Outliers”). In this blog, we even see brain or psychological implications as in the two previous blog posts.

To be simplistic, there is nothing wrong today with higher education that a good dose of paid internship or similar experiential education program would not fix by developing a cadre of passionate, self-confident, powerful student consumers of knowledge -  in college and for the rest of their lives.

• 0 Comments

Bird Brains: A model for Expanding the Human Mind in College?

Shoshana Korman QC’12 and Jim Stellar

Shoshana works in the laboratory of Dr. Carolyn Pytte who comes out of the bird song neuroscience tradition.  Shoshana and I agreed to write this blog with two intertwined themes: Neuroscience of brain plasticity and her own experiences being an undergraduate research assistant.  I find both stories to be encouraging of the idea that we can build the student’s capacity in college education as well as teach important facts and theories that will be useful to a career.

So, to start us off, Shoshana, can you explain in simple terms what it is happening when we say new neurons are born in the brain of adult birds and find a useful place in the brain circuit that underlies the production of bird song?

The first paper in the field was by Goldman & Nottebohm (1983), but my lab PI, Carolyn, often likes to quote an article by Bailey & Kandel (1993), who maintain that one of the key principles of behavioral neuroscience is that experience can modify brain structure long after brain development is complete. One of the most dramatic changes in the postnatal-brain is neurogenesis, where entirely new neurons are made in the brain of an adult. This phenomenon is seen across the animal kingdom, not just in birds. For example, animals that learn new odors have increases in new neurons in their olfactory bulbs. Similarly, increased neurogenesis in a memory related brain area, the hippocampus, has been shown to occur during spatial learning in rodents.

Our lab uses the zebra finch songbird to study the relationship between song learning and the brain and, I specifically to look at the song motor pathway (in pink, below).

Here, new neurons project from nucleus a brain area called the HVC to nucleus RA, which contains the motor commands for singing. RA projects to the motor neurons that innervate the muscles of the syrinx, the vocal organ in birds. Within HVC, new neurons are incorporated throughout a bird’s life.  For more detail see a 2005 article by Nottebohm.

That is very cool.  Who would have thought birds would have so much to teach us about brain plasticity?  But it was here that the phenomenon of new neurons in adult brains was discovered.  Now, please talk about another part of the lab’s work where you related song performance to the survival of these new neurons in HVC.

First, I have to point out that even though neurons are continually replaced in HVC, the songs of the bird that are the direct output of this motor pathway are stable. This is amazing in itself as the song itself is highly stereotyped and probably has to be to be useful to attract mates. An analogy can be made to human speech, which is also very stereotyped and stable and has to be to communicate well within the language spoken.  Like children who learn a first language without training, zebra finches learn the song of their social group when they are little.  They start to sing it in maturity and then auditory feedback from their own singing in comparison to what they have previously learned teaches them to perform well own song.

The new neurons that are continuously incorporated into HVC somehow must be “taught” to fit in to the ongoing song.  That is presumed to happen through auditory feedback because making it impossible for the birds to hear their own song leads to more individual variability in song performance and that may be a result of these new neurons “messing up” the existing song motor program without the correcting feedback. 

What is particularly interesting, and what you mention it above, is a recent result in our lab. We found that if one interferes with song production by treating the vocal apparatus with the drug Botox in otherwise normal birds, these new neurons in the HVC area of the brain do not survive.  They die by a process neuroscientists call apoptosis, which also happens in development when more neurons than needed for a brain circuit are produced and the extra ones are “pruned” away. 

It is almost as if the new neurons somehow knew they were not contributing to the song and died. Of course, I do not mean that individual neurons think…they are just cells.  But somehow their survival seems to depend on how well the brain circuit does in performing. Of course, there are other ways in which the nervous system can change in adults following experience, including the classical formation of new synapses to change the connections between neurons and a recent discovery that the myelin insulation around neurons may change with firing to affect timing of signal conduction to other neurons.

But let’s switch topics a bit and ask about what might be the result of changes in your brain.  You joined this lab some time ago and now you have certain skills you can do that I hear you teach other younger undergraduates.  How does it feel to have achieved this mastery in the lab?  Do you ever wonder if new neurons in your brain are finding a home in some area because you are learning new skills?

Coming into the lab my Freshman year, I was incredibly intimidated by the amount of tasks and level of skill necessary to complete the various experiments. I remember observing my upperclassmen and graduate student colleagues with awe, wondering if I would be able to perform lab tasks with the same degree of expertise. Sure enough, over the years I have been able to fine-tune my own skills in the lab to the point where many of my undertakings have become second nature. What I find particularly interesting is the similarity of my mastery of lab techniques with the stereotyped proficiency of a zebra finch’s vocalizations. Just as I have been able to learn lab protocols and methodologies from my mentor Carolyn, so too, the juvenile songbird is able to absorb the song of his tutor (who we even refer to as the “mentor”). From research, we know that neurogenesis plays a large role in the learning of the zebra finch song. Therefore, I think it makes sense to wonder if my own learned motor behaviors have someone caused a rewiring in my brain. When I think back to my early undergraduate experience and compare it to the way I am now, I can’t help but conclude that neuronal plasticity has allowed for the modification of my behavior.

This last statement says it all.  There are a number of books and other findings we will continue to review in this blog, but that last statement that you could come so far, and so could many of us, by happily learning from experience is the point.  And it is just too cool that the word “mentor” is used in your song bird research and again by you to refer to this important aspect of your experience as a student.  In an era where people say education is broken, this is education that works.

• 1 Comment

Flow

Eric Miller QC ‘12 and Jim Stellar

In his book “Drive,” on intrinsic motivation in the workplace, Daniel Pink mentions Mihayl Csikszentmihalyi and discusses the concept of “Flow.”  To begin with Csikszentmihalyi, he was born in Hungary in 1935 during the most volatile period of the 20^th century; World War II was raging across Europe. Hungary was under a fierce Nazi occupation and was being devastated by Soviet troops. At the age of ten, he left the country with his mother and two brothers, unaware that in five months a majority of his family would be dead. Later, Csikszentmihalyi recalls his ten-year-old self thinking at the time, “There has got to be a better way than this.”  That experience later inspired him to search for what made life meaningful in the field of psychology and still later led to do studies focused on creativity and play. It was during there that he developed his famous concept of “Autotelic experiences” (self-goals) or what he came to call “flow.”

Flow is defined as a transcendental, “spiritual” state in which everything being done seems perfect; goals are clear, feedback is immediate, success is assured.Flow is a state of complete engagement, producing a degree of focus and satisfaction which far surpasses routine motivation. The concept began to gain traction in 1990 when Csikszentmihalyi wrote his first book on the topic; the exposure to a wider audience produced a following. It has become a constituent in many theories of spirituality, self-help, education and business.

Flow underlies a majority of Pink’s arguments for inspiring a new sense of intrinsic motivation in employees who are working in organizations which “goes against” human nature. People want to be engaged in their work, not merely compliant. People crave autonomy- the freedom to do what they need to do in their own way and at their own pace alongside people they want to work with. They seek opportunities for mastery- to engage in “goldilocks tasks” which are not too easy and not too difficult, goals they can reach while still feeling a sense of accomplishment and betterment. Though mastery is admittedly asymptote- close but never caught- it is that which makes it both frustrating and alluring, constantly worth striving for.

Most importantly, people seek purpose. They want to know that what they are doing matters, not just for now or for themselves, but for everyone, everywhere, for all time. That is the ultimate goal, the greatest pursuit. Without the urge for achievement, correlated with our personal efforts and methods, we would not be human. Mihaly Csikszentmihalyi would never have the opportunity to identify flow because it would never have existed. Flow is the key to rising above the need for survival and realizing self-actualization.

We have seen its existence in literature and movies, noticed it working out in the gym or performing duties on the job, observed it in the fervor of religious prayer. Its existence seems unquestionable. So what is its role in experiential education?

If colleges can design programs different from what Pink calls Motivation 2.0- the old system based on extrinsic rewards and punishments (e.g. classroom grades)- then perhaps they can create opportunities for flow. We agree with Pink that situations in which intrinsic motivation dominates are ones in which people, including students, have more potential for autonomy, mastery of a task, and working in sync with others. It is difficult in the classroom; the dedication to structure and fairness means little engagement and the possibility for failure is punishment rather than motivation. Rather those situations can powerfully occur in an internship, on a service-learning project, doing undergraduate research, or stepping into another culture/country on an abroad program.

When a student loses herself (flow is occurring) in a specific internship or community service moment, that experience can help clarify the choice of major (e.g. political science) and a career path (e.g. pre-law). The student only familiar with classroom experience cannot be as confident in their decisions. Flow is the positive side of the so-called “school of hard knocks” that most college students experience only as alumni. The sooner students are exposed to “real world” education the more engaged they will be in learning. Students can be passionate about their careers and put in the required hours Malcolm Gladwell calls for in his book “Outliers.” Where Alum and Roksa say colleges fail in their book “Academically Adrift” there can be achievement. Incorporating experiential education allows students to experience flow in the “real world” which compliments what happens in the classroom and curriculum. It allows upper education to be effective.

• 0 Comments

What is student leadership?

Andrew DeMasters QC12 and Jim Stellar

I (JS) remain interested in student leadership as a form of experiential learning right on campus.  But the question is what is a student leader? And what makes him or her tick?  To address those questions, I thought I would talk to one, someone I have known for a few years and who is making an important contribution to helping us raise our participation rates in the teacher evaluation process after it switched to on-line and suffered a classical drop in student participation rates. Oh…and Andrew writes his own blog.

Andrew, let’s begin at the beginning.  What is a student leader and why do you want to be one?

A student leader is not someone who walks around with a title. Rather what I have come to understand is that more specifically, it is about taking initiative that inspires others to do the same. I have had the unique experience of coming to understand this concept.  Ever since I could remember, I have always been in the active pursuit of trying to become part of a something more than myself. My sophomore year brought a lot of growth in the sense that I have to realize that having a title of “VP” or “senator” is meaningless unless you define it. My active engagement in trying not only understand myself but to look at what I was doing with a greater sense of purpose ultimately led me to become a student leader. In my time at Queens College, I have never held position in student government but I have made my best efforts to certainly change a campus I was invested in. Sometimes, without necessarily having “power” you can still make have friends, colleagues, supporters who realize the efforts you are trying to take to improve their lives as well. In my years here at QC, with the help of others, I am proud to be part of events such as Midnight Breakfast, War on Hate, Major/ Minor Fair, and more recently the Teacher Evaluation process. I believe that my continued experience has only been possible with the help of others who shared in the same vision as me. I do not say that I am a student leader. I often believe that I am the first follower of an idea and therefore the movement begins.

What do you think is the essential learning opportunity from being a student leader in or out of the student government?

 There only needs to be one significant trait: a willingness to take action. I believe that being involved outside the classroom is highly educational for several reasons. First, regardless of any organization, you meet fellow students from all walks of life, whether they grew up in a bad neighborhood or avoided a giant debt with an undergraduate degree. By having a firsthand experience of just meeting these people, this is simply a taste of what the world and working environment is like outside of college.  Second, you learn how to participate in meetings. By actively engaging within an organization you learn about the membership, member’s strengths and weaknesses, and what the purpose of the organization really is. Third, you learn how to actively engage yourself within your surrounding community through creating events that are run by students for students. What is a better feeling than that of helping to bring something awesome to your college that will be fun and beneficial? Fourth and finally, you really get to discover what your potential can be through active participation. I have found that remember your college campus and participating in it helped to shape the type of person I am today. I often find myself doing things that I never thought I could do, e.g, sitting in a room doing paperwork or having coffee with a fellow member in the dining hall.  I believe it is some these basic truths that you receive outside the classroom that are so beneficial. In a way its learning that is optional, not required in a class. I have a firm belief that participating in an organization and having a chance to make a difference can certainly go a longer way, far past the perfect 4.0 GPA student.

How do you see this tying back to your major field of study and what you hope to do (at this stage) for your career?

            I really see that the experiences I have had in college have provided me with tremendous insight and planning for my future. I plan on attending law school with a Juris Doctor in Public Interest Law. I am attracted to this field because I have a deep sense for being involved with government related work.  My college career as a student leader has given me important skills such as scheduling meetings, policy research, public speaking, event organizing, and more importantly how to take an idea and put it into action. As both a double major in Political Science and Sociology, I believe that I have learned a great deal about how to understand and work with people, how to make successful policy, and how government should work for it’s people.  I have taken two seminar classes in Political Science: NYC Politics and Immigration Law. Each has given different perspectives of what problems are facing our country as well as extensive background information.  In Sociology, I have taken courses such as Deviant Behavior and Mass Media Pop culture that have exposed me to real problems, how information is relayed, and how it affects society as a whole. It is my intent to bring the same passion that motivated me as a student leader to my next challenge. My experiences as a student leader have certainty opened up the doors for what it is I want to do for the rest of my life: serve the public, make a difference, and be happy. I could not have asked for a better place as Queens College to further develop my skills, become a student leader, and have unforgettable experiences that provided me with how life after college will be.

Notice the use of the word “passion” just above.  Passion makes people act.  Some think it improves learning.  Certainly it improves time-on-task and that makes learning better.  In today’s world where so much of higher education is under criticism for not delivering a better educated student into the next level of schooling or into the workforce, this is an important concept.  Why?  Because a passionate student educates themselves as well as consuming what the college/university offers.  So the academic facts-and-theory learning is better because the student sees the point.  But more than that, the student sees the value.  Remember the old saying, “You can lead a horse to water, but cannot make him drink.” that common wisdom has set for us?  Well experiences like being a student leader can fire that thirst which we are calling passion.  It can happen in the classroom, for sure.  But why not build programs outside the classroom that make it more likely to happen.  Why not let students help us build those programs?  Is that not experiential education itself?  Then we can leverage that passion with the curriculum, drive higher standards, enhance learning, and deliver the best educated student possible.

• 0 Comments

Undergraduate research can happen to anyone

Maryam Waheed QC’11 and Jim Stellar

There is probably no student at Queens whose undergraduate research topic is closer to the area of neuroscience that I studied (dopamine brain basis of cocaine addiction) for more than 30 years than Maryam.  Maybe this is why we became friends as I did not bring that laboratory to Queens College when I took the Provost’s job 2 years ago.  So, to this seems a perfect opportunity now to combine perspectives and examine how undergraduate research works.  To begin, I thought it best to let Maryam introduce herself.  So, my first set of questions are:  Who are you, how did you end up at Queens College, how did you get into research on dopamine, and what are your future plans?

I am a Muslim woman born in Pakistan and raised in the United States.  After High school I went to Rutgers University for the first semester of freshman year and soon found that it wasn’t for me. I wanted my college career to be in a smaller campus and living away from home was just not for me. So, I chose Queens, being that it was both small and close to home. I was introduced to the Ranaldi lab by my psychology professor who needed undergraduate research assistants in the lab she worked in. She got me interested the second she said, “rats.” I have never worked with animals under experimental conditions so I figured it may be something I might be interested in so I joined the lab in January of 2008. I didn’t necessarily choose to get into research on dopamine, it just happened. But, I am glad it did because it opened my eyes to the issue of cocaine addiction and more interestingly, the neuroscience behind it. I recently graduated from Queens College and I am going to start my first year as a Dental Student in the New York University College of Dentistry. I really hope to continue research at NYU.

This is great.  Now talk about what happened to you in the lab.  You changed countries and changed universities, so one could conclude that you are just a strong person, not intimidated by anything.  But I bet there is a story here about how doing research and amassing the technical skills and understanding of the experiments advanced your own confidence.  Talk about that effect if it happened.

Going to lab every week for 3-4 days was hard work. It really taught me how to stay on my feet. I had a schedule where I was assigned a certain amount of time that I had to give to running the experiment every morning. Sometimes, I had to be at lab 7 in the morning in order to finish in time for someone else to run their rats and then I rushed to class. We all had lab duties that needed to be done every week; sometimes they needed to be done twice a week. Mine, was to make equipment used for surgeries. This was considered the hardest and most unwanted job in the lab. Of course, I volunteered to take on this job because I like challenges. It taught me how to use my hands efficiently which to me seemed very beneficial for dentistry. The experiments were where most of my learning took place. Working with cocaine was a confidence booster in itself. Going into the running room every morning and working with operant chambers, cocaine and rats really made me feel like a scientist. Everything had to be exact, from the dosage of drug, to the rat’s home cage, to the time it spent in the experiment chamber. You know where I had the most fun?  Watching these rats self-administer cocaine! I remember once I stood in front of this one chamber (of course the rat couldn’t see me) to take a look at the rat’s behavior and I saw how motivated the rat was to get his next infusion of cocaine; he literally pressed the lever non-stop for 5 minutes. To me, that was amazing because the rat was willing to work so hard for even just the lowest dose of cocaine because to him it was reinforcing.

I cannot help but notice in this section you write the parallel between the rat working hard and you also working hard.  Of course, what we neuroscientists think is that the drug was stimulating the pleasure system.  You also seem very happy to take on hard tasks, arrive early in the lab, and enjoy the career development where an undergraduate feels like a scientist.  I think this is very important. So, I wonder if we could get you to comment on how important it was to you to “feel like a scientist” especially given the path you took to get to the lab.

Having the responsibilities that I had in the lab made me feel independent. Feeling independent is what really made me feel like a scientist. I had my own swipe card, which meant I had access to the lab at all times. I had access to the “drug fridge” which gave me the feeling of responsibility and trust. These are all attributes that I think define a scientist, along with the experiments and the daily running of rats of course. It was important for me to feel like a scientist because I think if I didn’t, I really had no point of being in the lab. Going to lab wasn’t just a requirement that I thought needed to be fulfilled. It was more. It was important for me to know why I was doing what I was doing. Injecting rats with cocaine every day wasn’t just done for the fun of it nor was it to enjoy the rats getting “high”. It was done to see the effects it had on their behavior and this was done in hopes to find an explanation for cocaine addiction in humans. It was done in hopes to find a “cure” for cocaine addiction. This is why feeling like a scientist was important to me and if I could somehow continue to instill this feeling in dental school, I wouldn’t let the opportunity pass.

Notice that authentic experiences create their own motivation.  It is not that College is inauthentic with its classroom format, but there is a contrived nature about it with its course syllabus to tell students where they are, predictable exams, presumption that nothing on a test will not have been somehow presented in class, and even the fact that students sit in the audience and the professor … well, professes.  Additionally, it is what we know.  So the real-world experience of an internship is different, like the first job after college.  No one has a syllabus for a job and the “exams” can come at anytime the client or the boss asks a question or something happens.  Oddly, working in a professor’s research lab has the same real-world qualities.  And you see the reaction above. Students love to be trusted with real responsibility and generally react well, nicely complementing their academic learning and enhancing their career development in college, like going to dental school at NYU.

Another point that we want to make implicitly is that this kind of interaction is open to everyone and can have particularly strong effects on ethnic/religious/gender groups where they might not feel such a path exists.  This is why Colleges, their administration, faculty, and particularly students, must reach out to make sure all are included in experiential education the way we also must do in classical academic education.  A great way to do that is having a role model.  And we have exactly that happening here. 

Finally, we notice that once a student does an internship or undergraduate research project or something like that outside the classroom, they often also want to help the institution operate and not act like some manufactured product (e.g. a ball bearing in a ball bearing factory just riding through the process) but to help run the institution.  We need them in higher education.

• 1 Comment

Minority recruiting is experiential education for both

Govinda Davis NU’07 and Jim Stellar

Govinda was a student at Northeastern University who, at the end of my time there as Dean, was the head of a student organization of some years duration called LEAD (Linking Education and Diversity).  Over the years LEAD worked closely with the Admissions Office to bring in students of color from local Boston high schools, and eventually the University was able to give the Boston Public School students full scholarships.  Northeastern University was rapidly rising in reputation and national reach, which naturally attracted many great candidates for admission from all over the Nation and the world.  And the idea behind the LEAD program was to increase student representation from Boston High Schools and work toward a diverse student population. After Northeaster, Govinda went on to Law School at Seton Hall Law in New Jersey. She is now participating in the school’s entertainment law concentration with focus on intellectual property and copyright law in the music business. 

Govinda, as a minority woman yourself, start off by telling us what motivated you to put in a great deal of time to LEAD back in the day?

One of the reasons why I dedicated so much time to LEAD was because it was a great cause. I thought it was extremely important to build mentoring relationships with students who are looking to further themselves through education. I remembered how I felt as a high school senior and not really knowing who to ask about my SAT scores or how many colleges to apply to. I did have my parents and high school guidance counselors, however, it would have been great to have someone to talk to who had just went through the process and could speak to some of the real questions that I had about college life and what to expect. I enjoyed organizing the LEAD program, however I really loved the relationships that I had with the high school students involved. Now, while in law school I find myself in a similar mentor role. Although, my participation is not as in depth as it was in LEAD, I act as a mentor to prospective students and first year students here at the law school. I believe that strong mentor relationships play an important role in academic, personal and professional growth.

It seems obvious that your personal commitment would be necessary to inspiring the students with whom you worked at LEAD, but I want you to talk about it.  Put yourself in their place and tell me what you think they might have gotten out of mentorship from someone like you in that program.  I hope that is not too weird.

I think the students in the LEAD program got the chance to speak with someone who was not much older than they were and talk to them about what college was really like and what to expect. I think the students benefitted from having someone there every week who genuinely cared about their progress and wanted to help them in any way that they could. I think it was also helpful for the diverse students in the program to see mentors from diverse backgrounds being role models and sharing their goals of higher education. In addition, most of the students were from large Boston public high schools where the teachers and guidance counselors were responsible for hundreds of students and could not spend much time helping each student individually. However, our mentor program was small and each mentor was only assigned 5 or 6 students so that each student could receive individualized help choosing colleges, filling out financial aid forms and writing a great college essay. In addition to the formal workshops, I think students also built great friendships with their mentors and with the other high school students in the program.

Thank you.  How important is the positivity in the mentoring relationship?  You were always a very positive person in these and every interaction.

I think that it is very important to be positive and open to all types of mentoring relationships. You never know who you are going to meet and what type if information they’ll be able to give you or the great relationships you are able to form. The best advice is to keep an open mind and be receptive to what mentors are willing to share with you. There are different types of mentors. Some people sign up for programs like LEAD or sign up for mentors through their work or school. In those situations people are trying to make the mentor relationship work but there are also accidental mentor/mentee relationship which grow out of a two people who have something in common and get along very well. Every mentor relationship is different but I think people genuinely want to help others if they can and if a mentor actually finds someone who listens to their advice and takes what they are saying to heart both parties can mutually benefit from a mentor relationship.

You used a word above, “heart,” that I think carries special meaning in higher education.  Richard Light wrote a famous book “Making the Most of College: Students Speak Their Minds” in 2001 that makes the same point that college students really want mentors.  I used to stop there with my thinking, but now I see a connection to the other lobe of the brain in that famous Blaise Pascal 17^th Century quote that I often cite “The heart has reasons of which reason does not know.”  So, I would argue that it is not the logical mind of which we are so conscious and which education imbues with facts and theories that needs mentoring.  It is what a recent author, Shankar Vedantam, called “The Hidden Brain” – that part of the brain that stores our gut-level decisions, going back to the founding blog based on the book by Damasio that gave rise to the blog’s title “The Other Lobe of the Brain.”  This is why it is so important to have a student from a diverse background have at least some mentors who are from that background (as you also say).  People make decisions that they can succeed in College, that College offers a path forward, that this is where they should put their time and energy.  They do that on their own, but also with others.  It becomes like family and that can make all the difference, as you also say, to both parties.

• 1 Comment