Students using Facebook in your class? Better try a bit harder

41 Comments

By now, the idea that university students would spend much of their time in class on their laptops and tablets, browsing Facebook or the web, hardly seems surprising — and a recent editorial in the Harvard Crimson student newspaper confirms that this is the case. But the writer also makes an interesting argument, which is that the amount of time students spend online in class (which he calls the “Facebook Index”) is directly related to the quality of their professor. Since the web provides such easy access not just to social media but information of all kinds, he argues, teachers need to try harder than they have in the past to add value.

This might seem like a bit of a stretch for some, and in fact I got some substantial pushback from a number of people when I posted a link to the Crimson editorial on Twitter — including many who felt that it was unfair to expect even the most engaging professor to compete with Facebook or Twitter or text messaging. Surely most of these students are just wasting time on these social networks, or doing the modern equivalent of passing notes in class, rather than trying to educate themselves about important matters? After all, how much valuable insight could Facebook possibly have on things like the impact of Chinese trade policy during the Han dynasty?

It’s not just boredom — it’s competing sources of information

But Hemi Gandhi isn’t just talking about Facebook — although he calls it the “Facebook Index,” it’s about how much time students spend online in a variety of ways, including searching the web for information related to the course they are in. And since he was curious about why students at one of America’s most prestigious universities would waste their time online while in class, he asked some of them why they do it. Their answers were instructive, and more than a little frightening, at least if you happen to be a university professor; most said that they do this because:

  • The professor starts saying exactly the same things that appear in the textbook
  • Their instructor is confusing, and so they are looking for more information
  • The professor starts off on a random tangent that is not interesting or relevant
  • Students need a break from the topic to re-focus, or they are in a hurry and multitasking

Many of these responses will sound familiar to anyone who has been to a conference or a symposium where people are presenting PowerPoints, etc. There is no quicker way to lose your audience than to start reading from your slides, go off on a rambling tangent (unless it is amusing or insightful) or overwhelm your listeners with minutiae. In the past, people — including students — would have to sit silently while this happened, but thanks to laptops and mobile phones and ubiquitous Internet access, they no longer have to do so. In other words, you have to try a lot harder to hold a room.

But Gandhi makes a more interesting argument as well, which is that the web is disrupting higher education in more fundamental ways (something Harvard professor and author Clay Christensen also argues), because it is chipping away at the information-gatekeeper status that professors used to have. Just as the web is making it easier for people to compete with professional journalists, so it gives students the ability to find information themselves — and in some cases that information may be more valuable than what they are getting from their professors. Says Gandhi:

[T]he IT revolution has destroyed the traditional professor-student knowledge hierarchy. Access to knowledge has become easier. In the past, professors were knowledge gatekeepers when lecturing at the pulpit… but today, much of knowledge has become commoditized on the web.

Knowledge in any subject — semiconductor fabrication, Kantian logic, or exchange rate policy — can be accessed through a quick Google search. Online sources like Khan Academy, MIT OpenCourseWare, Wikipedia, and Google books are all freely and instantly available online.

Just like it has with media, the web is disrupting education

Gandhi argues that Facebook is just part of this much larger paradigm shift in the way that knowledge is transmitted in our digital society, and that Harvard and other universities have to respond to this and adapt if they want to remain relevant. Professors “need to realize that they are in constant competition for students’ time and attention,” the Crimson editorial writer says, and have to start thinking of themselves as “service providers who must constantly innovate to serve students better” by appealing to their students’ curiosity and their desire to learn outside of the traditional curriculum.

In some cases, adapting to a digital world can actually improve what happens in the classroom, as my GigaOM colleague Ryan Kim noted in a post about using social tools at school. He argued that using the web and social tools in particular can make it easier to appeal to some students who might not otherwise get involved in a classroom discussion (due to shyness or various other personal or cultural factors). Have any university professors thought about trying to do that, I wonder, or are they just assuming that most of their students are online because they are intellectually lazy?

I think Gandhi makes a pretty compelling argument that teachers of all kinds have to realize they aren’t the only ones competing for a student’s attention, and also that whatever information they impart can be fact-checked and assessed almost instantaneously. And if they are smart, they will try to find ways of incorporating the online world into their classes, rather than trying to outlaw it.

If the Crimson editorial writer was in my class, I would give him an A+ — regardless of whether he was surfing the Internet during my lectures or not. For more on disruption of education, media and other aspects of our digital society, please join us at GigaOM’s new RoadMap conference on November 10 in San Francisco.

Post and thumbnail photos courtesy of Flickr users Todd Clement and Jeremy Mates

41 Comments

Michael Moody

I am a Finance major and I personally use the internet everyday in class. Some of my classes the professors are great and I only use my computer to take notes and occasionally look up supplementary material. But some of my classes the professor just reads a powerpoint to us, that is also posted online, and is basically just a copy and paste outline of the book that I have already read, so in those cases I do sometimes wander over to Facebook and occasionally even Netflix (with captions of course).

Personally the internet has not hurt my grade at all in any of my classes. I still have a 4.0. I actually think that sometimes when I am back there watching Netflix I am actually helping myself, as in the case of my Psychology professor, anything that comes out of her mouth just confuses me. So, I stick to studying outside of class. I would skip the class but we have mandatory attendance policies.

The internet can be useful as a means of information retrieval, but as in the case of my Psychology class, I am pretty sure it has kept me out of trouble and from just sleeping during the class.

Higher education is so hit-or-miss. I have yet to find an OK professor. They are either great or just completely horrendous.

Rob

I’ll grant you that it’s definitely true that many profs commit the errors suggested by those student comments. Being tempted to check Facebook might be a symptom of a bad prof, but in just as many cases it could also be a symptom of a bad student. There might be some value in resisting the constant temptations that technological advances have brought us; recent research suggests that willpower, GPA, and achievement in both school and life are correlated, if not always in rock-solid fashion (see the recent book Willpower). I’ve sat in on large lectures, and for people with laptops in those classes, and I’d estimate that it’s about 20-80, 30-70 between people checking Facebook & people using their computers productively. Most of them are working hard & paying attention; as a TA, I also can say that the Facebook-checkers definitely aren’t the ones that are doing well. Listening to a lecture is difficult, but doing difficult things can also make us better. (It’s important to note, though, that if the prof really sucks, you may not get much better; let’s not be so black & white here.)

The ‘customer service’ model of education, however, is definitely a misleading one: if you have to push people beyond their limits to do better and then evaluate them, you’re doing something quite different from serving someone a martini with a smile or updating their productivity software. See the study “Academically Adrift” to see the fruits of our current ‘customer’ focus in education: what you get is less demanding work, less time spent studying, and little learning. It’s cute to talk, hypothetically, about how awesome it would be if profs were somehow always as fun and interesting as Facebook, but on some level there’s wishful thinking involved. Education will always involve some form of difficulty if it’s doing its job. I’ll grant that you’ve got a point insofar as the key is to make sure it’s the right kind of difficulty.

It’s also true that most people who haven’t taught vastly underestimate how difficult and time-consuming it is to put together a good presentation day-in & day-out; even if college profs were to put in 50 hour weeks on teaching & prep alone, in addition to say 20+ hours on research & administration, you’d still probably end up w/ very few awe-inspiring teachers simply because the ability to consistently wow a crowd is a rare skill. There simply aren’t enough people with that much skill willing to work those kinds of hours with that much stress for $50-70k/year–or $20-30k/yr with no benefits, if you’re talking about the adjunct professors who make up roughly half of college instructors. I frequently see people who keep wishing for amazing teachers to simply appear, but little regarding how we are going to make them appear. Instead of writing editorials about how we’re bored in class, why not do some research on the issues involved? Not to absolve professors from the responsibility of trying hard to be engaging educators, but simply telling people to “try harder” misses the larger issues at play here.

Don Trey Korte

As a high school teacher, there’s a huge difference between college students looking for supplemental material in classes they choose to take and a bored teenager instant messaging friends during a required class. I find Facebook and cellphone is the biggest detriment to student learning, not when I’m teaching material, but when students are expected to use class time to work and complete projects. Not all students have the maturity or self-discipline to use the technology appropriately. There’s also the matter of it just being plain bad manners to be texting, posting on Facebook, or surfing the web while someone is giving a speech or a presentation. It really bothers me when students will text while other students are presenting. The problem with this article is that it assumes that all students are in class, eager to learn, and the only reason they might resort to electronic media is the instructor isn’t engaging enough. Many of the students, at least in the K-12 setting, don’t want to be there. I don’t think most instructors are concerned about the highly-motivated, high-achieving students who are using technology to further their education. I’m concerned about the students at the other extreme where it’s clear the technology is another distraction impeding success.

Brooks

I agree with this article in most places. As a student at a major university, its very common to see computers in a class that is considered to be taught by a bad proffessor, whereas an interesting class will have far less computers. That being said often times i will use my computer to check something I think was interesting a lot. Its very useful and it helps me keep interest

Munia

As a university professor in the area of foreign languages, I find the use of the internet, facebook, google, etc, very disrupting. Usually the students are using all those media in English while I am teaching them Spanish in Spanish. It is worse if they are sending text messages, because they are texting in English while I am trying to make them write in Spanish. I can see the “short circuit” in their brains when I ask them questions in Spanish and they reply in English because they are “gone”, thinking in English. Usually, facebooking, or what they are looking on the internet is not related to Spanish. . If I am boring or not as a professor, it can be, although I do a lot of activities that are very engaging (students have to circulate around the class, talk to each other in Spanish finding information about classmates, activities they do, their friends do, etc). The challenge may be to think in a different language, which IS a challenge. On the other hand,the classes I teach are a mixture, a sort of blended class, they do homework online at their dorms, practicing grammar and vocabulary. When they come to class they are expected to use the language to communicate, to actually speak in Spanish. If they are texting, facebooking, they just do not engage in the classroom activities because the are doing all the online stuff in English. Also, you cannot make someone to speak or interact if they are “connected” or “glued” to the screen. I banned the use of laptops and computers in class, also the use of the cell phone and I give zero in participation if the student uses the cell phone or goes to FB. I allow the use of laptop at 400 level classes for taking notes and exploring further and usually the students (they know me) ask first if they can bring their laptops or not. Students at 100 and 200 level, even at 300 level also have the tendency to use online translators in class and, normally, online translators do not work, mainly when what the student is trying to translate is an idiom. Then, the use of internet in the field of foreign languages do affects the level and the accuracy of the knowledge the student is learning and the efficacy of what the instructor is trying to teach. Foreign language classes online are becoming a trend, though. Personally, I have had students that started with an online class, then took the next level in a regular class and failed. It depends, of course, on what the student is trying to do or is going to do with the language. If it is just to fulfill a Gen Ed. credit, it may work. If it is about minoring or majoring in a foreign language, it is not the best way to start learning a foreign language. Nothing can substitute the human interaction in this type of classes. (Disclaimer: I am not a native speaker of English, you may find grammatical mistakes in my comment).

Nick

I’m a high school student (year 9, I believe America calls that middle school?), and my science teacher that doesn’t seem to actually know much about his subject, instead referring us to the internet for everything. Whenever any work is set, he simply gives us a website without any explanation. When we have a practical task he hands out worksheets printed from the net.

Now I would be okay with this, except that the provided resources do not fully explain the task (as they are designed to) and the teacher himself can’t explain anything.

The one redeeming feature is that the tests he gives are simple multiple choice and some questions give away answers to others, but I’m worried I won’t know enough next year when I get a different teacher.

This is obviously a not so effective use of the web as a teaching resource.

Jae Vick

As a graduate student who uses a laptop because of a physical disability, I agree that the internet can often be a valuable tool. My computer helps me immensely in class, when my instructor is competing not only with the internet, but also with pain and medication for my attention. Using my laptop to look up something that can help clarify, or find a detail or anecdote that can cement the point in my mind is hugely helpful. I also hate sitting in class listening to the instructor or the teacher make things up or wander off topic because they cannot recall factual minutiae. I often look the detail up, tell the class, and then we can move on. Recently, however, I have had to fight (in a Library Science program, no less) to keep my laptop.

I do question pinning the “blame” on teachers for not being interesting enough. I don’t think computers in the classroom are a bad thing, so I don’t think anyone is to blame for it. I also think that instructors who are already against the use of computers will not change their minds because we accuse them of being lazy and uninteresting. On the same note, is disruption an academic term for this? I think of it as deconstruction, largely because disruption has such a bad connotation, specifically in a class.

Mr. E Mann

Out of the four classes I have, I’ve only used Facebook in one of them. And it’s generally when the professor isn’t making any sense. The GSI does most of the real teaching anyhow.

Nicole

definitely agreed.

In the classes where my teacher literally reads straight from the book, I’m dying without my laptop (she prohibited them in class)

In the classes that allow me my laptop but are incredibly interesting and providing information I can’t get from simply reading the slides, I pay much more attention, even though the world wide web is right in front of me.

AU_Junior

I would argue that the “information gatekeeper” role of a professor isn’t dying, only changing. Yes, every kid with a smartphone carries the sum of human knowledge in their pockets, but how do we identify patterns, repeated mistakes, or innovations? How do we piece the information together, and what do we make of it? Long before internet invaded the classroom, the best educators were those who not only delivered information, but helped students understand and interpret it.

BJ Roche

I regularly catch kids doing other people’s online courses in my computer lab writing class. Texting mom, e-mail, whatev. It’s like wack-a-mole. You have to ignore it or you just spend the whole class chiding 21-year-olds. This technology is a disaster for this generation. It used to be that you wanted a “wired classroom.” Now I wish I had an “unwired” one.

Julie Ann Brown

I have been using Facebook for approximately five years with my students…at their invitation! My marketing students were “positioned” to be social media marketers upon receipt of their marketing certificates before their were even many formal job descriptions created for this type of digital marketing career.

Julie Ann Brown

I have been using Facebook with my students for five years…at their invitation…it has changed the face of how I teach marketing…my students were social media experts and getting part-time jobs before the marketing place made it a relevant career. I love Facebook. Professor Julie Ann Brown, Santa Barbara City College

Patrick Chalmers

This is why I love the internet and its exploding opportunities for access – we’re finally approaching a democratisation of knowledge acquisition, 2,500 years after those pesky Greeks invented the idea of democracy.

This process of breaking the monopoly on knowledge is extremely healthy for our societies and promises enormous dividends for our collective creativity. Why anyway should one individual have a monopoly on how to structure university classes and what to teach? Why also does it cost so much to join the conversation just to pass a few exams and get a piece of paper at the end of it?

It’s not just university faculty members that need to be on their games either. Journalists face quite the same challenges from savvier audiences, as do politicians. This is where things get exciting – the major benefits I see from the revolution in internet and social media will come from the better journalism and better, more accountable governance that they force into being.

Probably not tomorrow, mind you.

Don Stanley

“In some cases, adapting to a digital world can actually improve what happens in the classroom”. I completely agree. I require students to use Twitter in my courses at UW-Madison and I encourage the use of technology (granted they are strategic communication courses). I’ve found this has increased engagement, helped steer class discussions and made discussions more rich. Students are contributors rather than receivers of information and they take more ownership of their classroom experience. I feel this has helped improve my teaching as well. We have guest lecturers Skype into class and the students can use Twitter to ask questions and engage the guests before and after class making which is wonderful. I know it isn’t always easy to adapt to change, but I find it’s worth the effort.

Lauren

I am currently attending university and last semester took an electronic commerce unit in which the lecturer used a variety of different online methods to keep us engaged, including getting students to use the vitural world of Second Life to give presentations to the rest of the class. It’s been my favourite unit by far and I always looked forward to going to class because my lecturer made it interesting to learn.

nicoleandmaggie

Yeah, I call bull-excrement. I was a student once. ICQ (esp from my boyfriend) was more interesting than anything any professor could say, especially in difficult courses. No matter how entertaining the prof. Would I have told a surveyor that at the time? Probably not. I probably would have come up with one of the excuses the kids have up there.

I would like to just say, “Choices equal consequences” and leave it at that. In the end it *is* the student’s responsibility to learn and if they choose not to, that’s their own problem. However, when students are actively not paying attention, that has negative spillovers to other students. When a kid is playing hearts, that creates a terrible environment for everybody and learning is less likely to occur for anyone in the class.

Boris Wertz

There are big opportunities in how technology and especially social media can transform education and it is a big investment focus of mine. One of the companies / investments I am most fond of in the education space is Edmodo, a communication platform to serve the K-12 space that has gotten huge traction in this past year as more and more teachers are adopting new communication tools for the classroom.

AnthonyP

I think Gandhi, and perhaps others, are missing a very fundamental point – there is a huge gap between information and knowledge, and the application of that information and knowledge. A good professor should be teaching the application, which includes the “w’s”: Who, when, where, what and perhaps most importantly, why. How to do something is often the least important piece of knowledge a person can have, if they don’t really understand why they need to do it or when it is appropriate or what direction to take.

Patrick

Thanks Matthew…great post as usual

 

There are of course some good points to the “Crimson article” and your commentary on it; you hardly need me to point them out.  I do have some points that I think are worth considering:

 

1) The Harvard Crimson’s writer might be correct; however, it largely hinges on the position: the primary criteria of being a good professor is engaging/entertaining.  If this is the case, then the greatest professor in my opinion is the Simpsons seasons 3-5.  Its kind of alarming that “being engaging” is not a criteria to judge your mechanic or doctor, but apparently completely justifiable when judging the person who trains them.  While there is something to be said about packaging your message and information competition due to the internet; hopefully, teachers add value beyond the stage show.

 

2) We could equally put the focus on the learner; they have to try harder to hold their own attention given that the world is more intimately distracting.  Learning, especially at the university level (and perhaps more so at Harvard), can be hard; students might have to push themselves to find the value and to try to stay focused; is that too much to ask of students at an institution that denies access to about 1500 high school valedictorians a year?  Postman has argued that the most dangerous things on TV were shows like Sesame Street that taught learning is fun and easy, instead of its fun to learn.  Huxley warned against pleasure and distractions as a way we can be controlled.  It seems to be happening – how much harder will it be if/when gamification really takes hold of our society (I wrote some more about this here: http://bit.ly/loXDhZ )?

 

3) The problem with doing the majority of learning through independent searches online is the difficulty of vetting the information.  Unless they make logical or internal factual errors, it is very hard to apply critical thinking in topics you know little about.  I forget who recently Tweeted (was it you?) to the effect: they agree with what they read online, unless its something they know about.  That is the part of my point on this post: http://bit.ly/sRawWp and it is the lesson we might learn from every site who talks about webpage design as a criteria to judge the soundness of websites. 

 

This discussion puts me in mind of ratemyteacher.ca where one of the 3 criteria for being a good teacher is “easiness”…is that what teacher’s should judge themselves by?

 

Patrick

Mathew Ingram

Thanks, Patrick — it may be unfortunate, but the reality is that teaching is to some extent a performance, and being engaging is going to be one of the criteria no matter how you slice it. Even Socrates knew that :-) And I agree that students have to face up to their responsibilities as well — and that we can’t necessarily depend on their abilities to find reliable information themselves. That’s why they need help from their teachers, and the web and social media are tools that can help with that.

Y.

Speaking as a student (sophomore at Columbia University), the important thing is a professor’s ability to clearly impart the necessary information WHILE being engaging/ entertaining. Personally, I do not bring a laptop to class, as I prefer to take notes by hand, but the fact is that sometimes class is so dull that I cannot bring myself to pay attention, and instead end up zoning out or nearly falling asleep even without technological distractions. The fact is that some people may be incredibly intelligent, but just aren’t capable of teaching. I think that we need to reexamine the university system and try to find a way to better serve the students while still achieving the other goals of universities; perhaps hiring two sets of faculty, one to teach and one to do research, might fix the problem. My best professor at Columbia thus far has been Sunil Gulati, who is primarily the president of the US Soccer Federation and lectures in economics on the side; meanwhile, I’ve had many horrendous professors who have made outstanding research contributions to their field.

Amanda Krauss

Agree with Mark on the need for a reality check here. Having taught for ten years, I second that students’ self-reports are not something we should treat as reliable factual information. And I’m all for doing away with gatekeepers, but there is also a need to recognize the poor information literacy most students bring into class with them.

Mathew Ingram

Thanks, Amanda — I agree that many students may not be telling the truth about why they go online in class, but undoubtedly some are. I think teachers could make use of the web to help their students learn, instead of only seeing it as a distraction or a time-waster.

Niclas

Good article and comments, just I believe a notion about the difference between the quality of information would be appropriate. Google all you want but don’t expect the higher levels of understanding lectures and teaching can provide.

I remember a simplified but useful hierarchy; data < information < knowledge < expertise. Google seldom gives you anything more than information, though, as I recall, lectures weren't a safe bet by any means :-)

Rick Fletcher

I teach a 300 student class at a top 100 research state U. This year I encouraged use of laptops and smart phones as feedback devices for in-class polling. Exam scores have been 1.5 standard deviations below a 5000 student average. I make use of the devices 3-4 times per lecture but the devices provide too much distraction the rest of the time. I’ve also had numerous complaints from students this year specifically about their neighbor’s browsing distraction. I use texting, Facebook and Twitter for discussion and communication but I’ve been forced to discourage in-class use. FWIW, I’m stuck teaching this class because I’m among the most engaging teachers in my college-I’m up and down the aisles, questioning, discussing, encouraging and so on. The results of this expt were so severe, I’m convinced and ended things mid semester. It sounds like the student comments are rationalization.

Nora Paul

Rick – your experience matches one that my colleague discovered. She had had students tweeting comments about the information covered in the class – even making their contributions part of the points earned in class – but found that test grades dropped. She discontinued the practice. Another problem was some students used the tweets to say snarky comments about other students tweets or in-class comments. She was afraid these online instant reviews of what people said in class would have a further chilling effect on people speaking up.

Ryan Craft

Interesting Rick. So do you prevent students from using laptops in your class? Or have you just stopped actively encouraging the use for in-class polling?

Ryan Craft

Great points. I believe that professors who actually embrace the use of laptops and social media in the classroom, as opposed to fighting it, will see the most benefit.

One example of an effective use of social media (Twitter) in the classroom is @2450fall2011. With a class size of over 200, discussion and conversation is generally thought of as impossible. However, with the help of GroupTweet this professor is allowing the entire class to consolidate the students’ discussions/reactions under a single Twitter account. Comments and discussion can take place in real-time during the class or even help to extend the conversation outside of the classroom.

Goverment Gripes

Fellow Citizen as we move into the rough and tumble of the campaign year ahead, politicians need to drop what we have lived by over the last 50 yrs and look at solutions creatively, without boundaries. Please take a min out of your Facebook, Twitter or Blog time and Post a Gripe or Solution real-time at https://www.governmentgripes.org

Mark Hamilton

I agree with a lot of what’s in this piece, but it’s also left me a little steamed. I have students who are never online during class, I have students who are online and augmenting whatever we’re talking about (through a quick research hit), and I have a small number of students who are online and totally disengaged with everything happening in the classroom. Attempts to involve them in discussion fall flat; mediocre grades don’t move them away from the screen. I’ll take the blame for my failings as a teacher, but conclusions based on students responses that don’t include any responsibility on their part for their own education, miss some of the realities of the classroom.

Mathew Ingram

I think that’s likely the case in lots of classrooms, Mark — there are some kids who will never engage or take part in their own education because they just don’t want to be there.

Allison

As a college instructor, I can always tell when it’s Facebook or something related to class. When it’s the former, I call them out– it’s so distracting. But, I don’t fight the beast by banning laptops– I was the type of student who preferred to type instead of write my notes by hand, so I get it. But often when I teach I’ll have a question myself and will then ask a student to look something up for me, right then and there in the middle of my lecture. Other times, they’ll do the same and add to my lectures when they have questions. It’s the only way I can think to integrate and keep their attention. Though I admit that even *I* can be boring. Sigh.

Total

Students never just goof off in class? I’d suspect that some of your analysis is correct, but I’d also suspect that a lot of students simply are too used to the immediately interruptable life to stay focused for too long.

John E. Bredehoft

Disruption, or supplementation? It could be that the professor sparks something in the student’s mind, and the student is suddenly eager to explore the spark further.

Mathew Ingram

That’s a great point, John — I totally agree that could also explain a lot of internet use in class, to seek supporting information or clarify something or just to explore. Thanks for the comment.

Justin H

I’m a third year law student at a top-50 ranked law school and am constantly online digging deeper into case law and finding out the context of certain cases. Professors often assume that the background knowledge they possess is the same background knowledge their students possess, but as a non-californian studying at a California law school, I don’t remember or know a lot about Gray Davis, or Ronald Reagan as a Governnor, or Willie Brown as Speaker of the Assembly, yet all three of these political figures are extremely important to understanding the context of California law. The Internet is a great tool for education when used properly.

C Coleman

I am a college student who almost constantly has the internet open during lectures, and you’re right on. I’m almost always using it to look up points from the lecture that interest me and to give greater depth and context to what I’m learning. It’s a great resource, and I feel that I get much more out of lectures.

Comments are closed.