Abstract
Introduction
Methodology
Results
Discussion
Conclusion
References
Footnotes
Appendix A
Appendix B

Teachers’ conceptions of the Internet and the World Wide Web:
A Representational Toolkit as a model of expertise*

James A. Levin
University of Illinois at Urbana-Champaign

Matthew J. Stuve
University of Northern Iowa

Michael J. Jacobson
University of Georgia

1999, Journal of Educational Computing Research, 21(1), 1-23.


Abstract

Given rapid changes in the use of network computer technologies in education, it is increasingly important to better understand teachers’ conceptions of these technologies, especially as used in teaching and learning. This is particularly true regarding teachers’ ways of thinking about the Internet and the World Wide Web. Two questions are addressed in this study: First, does a teacher’s conceptual representation of a network technology have an impact on how he or she uses it? Second, do experts think about network technologies such as the Internet and the World Wide Web in ways different than novices? To examine the role of teachers’ conceptual representations in detail, a survey and ten case studies were conducted among pre- and in-service teachers enrolled in university courses. The survey elicited the representations that teachers had of the Internet and the World Wide Web. The case studies consisted of network problem-solving tasks using think-aloud protocols. The results show a surprisingly diverse set of plausible representations of the Internet and the Web, not related to the level of technical expertise. The case studies reveal relationships between representations and navigation strategies, in which experts employ multiple structurally and epistemically different representations at different stages in network tasks. This suggests that experts have a "representational toolkit" containing many different representations. Experts not only know how to use each representational tool, but they know which representational tool to use for which purpose and how to sequence the selection and use of multiple representational tools. This reconceptualization of the nature of expertise has implications for the design of educational applications of technologies, for teacher training, and for learning and teaching more generally.

 


In 1992, the Center for Technology and Education conducted a national survey of K-12 telecommunications. One important outcome of this survey was the need to better understand teachers’ conceptions of technology and its uses. Honey and McMillan followed up this survey with in-depth interviews of 18 teachers, and found different "conceptual representations" ("information superhighway" vs. "information ocean") among the teachers. Previous to these studies there had been little research in how different conceptual representations might impact network usage, navigation, and learning. Research of this type is vital if we are to understand how teachers and students can use network learning environments productively.

We are interested in the conceptual representations users have for using computer networks. Our definition of conceptual representations draws from theories of mental models . We have extended the theories of representations and mental models to that of more general "conceptual representations," as those internal, often informal and naive, representations that guide the user’s actions on a task, creation, or problem. Conceptual representations often include visual imagery of the system to explain its function, or they can be metaphorical explanations as to its meaning. Together, these features of conceptual representations provide guidelines for acting, especially when no explicit strategies exist .

The Internet and, more specifically, the World Wide Web (henceforth Web) are still new educational media for many teachers. While many if not most teachers have browsed the Web, few have taken the next critical steps, such as using the Web productively in classroom learning activities and creating Web resources with their students. These steps require not only profound changes in practice for most teachers, as is typical of other classroom computing practices , but also changes in the images of teaching that teachers hold with respect to their practice . Furthermore, we have witnessed how teachers struggle with the sometimes complex tasks of searching, navigating, downloading, and uploading in a medium that designers of Web-based educational materials take for granted.

The goal of this study was not to find the most plausible representations that explain the Internet or the Web, but to empirically collect and analyze the conceptual representations that teachers have and to determine how teachers employ those representations in systematic ways in network tasks. It is also our goal to isolate and describe characteristics of those representations and their adaptability to problem solving and network navigation. If a particular representation predicts performance on a relevant task, or if it is used by a significant number of people, then it might be used as a basis for training and for designing network learning environments.

In order to examine the role of these conceptual representations in more detail, we conducted a series of studies of how teachers think about the Internet and the World Wide Web. These studies included surveys and case studies of network problem solving using think-aloud protocols. This paper will address the following questions: (a) Does a teacher’s conceptual representation have an impact on network use? and (b) Do experts think about technologies such as the Internet and the World Wide Web in ways different than novices? The terms "expert" and "novice" pertain to their expertise with technology, not teaching. The answers to these two questions have implications for the design and implementation of educational Web sites in terms of teacher training, navigation, and, ultimately, use in the classroom.


Methodology

This is a qualitative study using data from surveys, interviews, and observations conducted over a two-year period. We identified three variables initially: (a) the experience of the subjects; (b) the exposure to network activity over time; and (c) the multi-dimensional robustness of individual representations as they relate to actual use of the Internet in the context of specific tasks. We sought to categorize the technology expertise of the subjects in terms of previous technical skills, familiarity with the Internet and telecommunications, and subsequent change in technical expertise following exposure to network tasks.

The subjects were pre- and in-service teachers from the University of Illinois at Urbana-Champaign (UIUC) and the University of California, San Diego (UCSD). Other volunteers from UIUC who were not pre- or in-service teachers were also studied in order to establish a more descriptive model of technical expertise. Three instruments were used to collect data. The first was a survey to elicit self-descriptions of conceptual representations from the subjects (see Appendix A). Before eliciting the individual’s representations, we first provided an example to the subjects:

"For example, people often think about the atom as being like a miniature solar system. The solar system is then a mental model of the atom."1

As shown in data from this study, this analogy was usually sufficient for eliciting statements of conceptual representations from the subjects. The survey was administered at the beginning and end of the courses (we will refer to this as "pre- and post-exposure to the Internet") and included interval scale assessments of the subjects’ computer and network experiences and as well as their teaching experience. The survey included open-ended questions about conceptual representations of the Internet, E-mail, and World Wide Web.

The survey data helped us find preliminary answers to our questions and to construct the task analysis for the case studies. This task analysis was the second instrument, which focused on the problem solving behavior of subjects engaged in tasks using the Web (see Appendix B). To describe and analyze the ways that the subjects employed conceptual representations during the tasks, we developed a think-aloud protocol that was informed by methodological frameworks originating with Newell and Simon and operationalized by Ericsson and Simon . The tasks were designed to reflect typical ones that teachers might encounter for which they would use the Internet to find resources. The tasks are described in Appendix B.

A pilot study was conducted in the Spring of 1995 during the first wave of the growth of the Web. Teacher education students enrolled in courses at UIUC responded to a survey of their conceptual representations. Data from that pilot study were used to refine the survey and design a think-aloud protocol for observing subjects during a network problem-solving task.

From the pilot study, we discovered a systematic problem with the phrasing of the questions. We were concerned that the subjects would find terms like "conceptual representation" difficult to understand, so we used the term "image." However, this generated unpredictable responses. When we asked "What is your image of the Internet?," some subjects responded "Positive" — interpreting the question as referring to their opinion of the Internet. As a result, we re-worded the probe question using the term "mental model" and we provided an example of one (the solar system as a model of the atom). This revised survey was used during the Summer and Fall of 1995 with three subject groups selected for their levels of network experience.

The first group was a mathematics education class (CI 336) at UIUC. The students in that class were mostly in-service math teachers working towards a master of education degree during the Summer 1995 session. As part of this course, the students were provided with laptop computers equipped with Internet applications. One of the goals of this course was to expose the teachers to mathematics education resources on the Internet. The second group was a teacher education class (TEP 231) at UCSD. The students in the class were all practicing K-12 teachers enrolled in a master of education program at UCSD. The class focused on technology and education, including e-mail and the Web. TEP 231 also took place during the Summer 1995 session. The third group was a graduate course on educational computing (EDPSY 387) at UIUC during the Fall of 1995 that heavily used the Web and e-mail. The survey was repeated at the end of each course, but without the questions regarding self-evaluation of skills. In all, 61 subjects from the three classes responded to at least one survey.

For the case studies, volunteers were solicited from each class, as well as from expert network users within the College of Education at UIUC. Each subject was informed that the interview would be recorded on audio tape and that it would take less than an hour.

The data were analyzed along emergent dimensions. First, we tried to classify the conceptual representations as a whole, but their diversity made this problematic. Later, we focused more on describing that diversity of representations. Second, by gathering self-assessments of experience, we were able to examine the representations in relation to expertise. Third, by gathering pre- and post-exposure data, we were able to analyze the dynamic attributes of the conceptual representations over time. Fourth, using the data from the task analysis, we examined in more detail how individual subjects employed their representations. We also considered the extensibility of the representations to other settings as well as the epistemic qualities of the representations.

Results

Survey Findings

Responses to the self-evaluation questions were converted into scalar indices to determine an overall measure of technical expertise. This is referred to as the Expertise Factor and is equal to the average of the six expertise questions (experience with computers in general, networks, e-mail, newsgroups, gopher, and Web). This measure was used to rate the subjects’ overall computer and network expertise into three classes: novice, intermediate, and expert.

With respect to the questions about conceptual representations, not all subjects responded to all the questions. Some subjects drew diagrams of the representations, while others pointed to earlier responses. We decided to focus just on the Internet and Web responses since many of the subjects had significant experience with e-mail. Responses from all three subject groups were collapsed into a single matrix. Table 1 shows frequencies of responses for each group for the pre- and post-exposure surveys.

In Table 1, Non-responses include both blank responses and those explicitly not conceptual representations (responses of "don’t know", "?", etc.). No-data in the post-exposure column indicates subject dropout. No-data in the pre-exposure column indicates subjects for whom full data were collected only for the post-exposure (i.e. late enrollment).

Table 1
Frequencies of Responses for Each Group for the Pre- and Post-Exposure Surveys

Pre-Exposure

Post-Exposure

Internet

E-mail

Web

Internet

E-mail

Web

CI 336

9

9

8

7

7

7

EDPSY 387

15

16

17

12

10

12

TEP 231

18

17

15

18

18

19

Total Representations

42

42

40

37

35

38

Total Non-Responses

15

15

17

4

6

3

Total No-Data

4

4

4

20

20

20

Total Responses

61

61

61

61

61

61

 

Diversity of conceptual representations. The most striking result from the survey data was the diversity of conceptual representations. The responses were shortened into a few-word statement about the representations. Tables 2 and 3 are summaries of all the unique responses to the Internet and Web questions, respectively, that could be interpreted as representations.

Table 2
Internet Conceptual Representations

Count

Pre-Exposure Representations

Count

Post-Exposure Representations

7

Web

6

highway

6

highway, interstates

6

Web, spider Web

3

communication

2

encyclopedia

2

brain

1

city

2

library

1

community

3

phone calls, lines

1

computer chip

2

solar system

1

connection of information

2

universe

1

filing cabinet

1

BBS

1

foggy world

1

encyclopedia

1

fungus

1

filing cabinet

1

funnel

1

fishnet

1

Holodek

1

fungus with tentacles

1

lattice - Interconnected

1

lattice

1

library

1

maze

1

network of clients and servers

1

nervous system

1

neural networks

1

schema

1

octopus

1

sea

1

solar system

1

Star Trek Enterprise

1

teleconnections

1

toy jacks

1

telephone system

1

tree

1

water molecule

1

wave - interactive

1

wave - surfing

39

22 Total Unique Responses

32

22 Total Unique Responses

 

Table 3
Web Conceptual Representations

Count

Pre-Exposure Representations

Count

Post-Exposure Representations

12

Web, spider Web

5

Web, spider Web

6

library

2

freeway, highway

2

BBS

2

information flow, sharing

2

information, storerooms

1

ant trails

2

link - international

1

BBS

2

multimedia book, lines

1

city

2

streets, roads, highways

1

communication

1

access to Internet

1

community

1

brain synapses

1

conference room w/ encyclopedias

1

encyclopedia

1

connected computers

1

fishnet

1

fungus

1

fungus

1

funnels - visible

1

graphical Internet

1

global village

1

homebase

1

grain of sand

1

jacks

1

library - floating pages

1

nervous system

1

nebula - amorphous

1

solar system

1

network operating system

1

visual representation of culture

1

neural networks

1

publishing

1

Star Trek computer

1

supermarket of ideas

1

supper table

1

tangle of connections

1

telephone system

1

TV channel

1

USA Today

1

window on the world

39

18 Total Unique Responses

33

27 Total Unique Responses

 

As shown in Tables 2 and 3, there are similarities in how subjects perceive the Web and the Internet. For some, the Web and the Internet were the same concept. This was the case in 1995, in the early days of wide-spread use of the Web — this conceptual overlap is probably even more widespread today, since many people are now introduced to the Internet through using the Web. For others who were more expert, there are differences in the representations of the two. "Web" (and its variants) was the modal response for the Internet question pre and post. It was expected that Web would be the modal response for the Web question, as Table 3 shows. What is interesting in Table 3 is how the subjects discovered for themselves more salient representations for the Web than just "Web." "Web" is still the modal response in the post column with 5 responses, but this is down from 12 responses in the pre-exposure column.

Expert versus novice representations. Four cells of data from Table 1 were expanded into Table 4 to show the relationship of expertise to the existence of a representation. Table 4 shows the frequency of responses in which the subject replied that he or she did not have any representation, or that the question was not answered. These data are referred to as non-responses. The non-responses to the Internet and Web questions were sorted by expertise (novice, intermediate, and expert). Table 4 confirms an expected result in which exposure to network activity allows subjects to generate representations (e.g. there were fifteen non-responses for Internet before exposure vs. four afterward). More importantly, Table 4 shows a pattern that indicates that novices are less likely to report having a representation than more experienced users.

Table 4
Percentages of Non-Responses

Internet

Web

Pre

%

Post

%

Pre

%

Post

%

Novice

8

40%

1

9%

Novice

10

50%

1

9%

Intermediate

5

20%

2

10%

Intermediate

4

16%

1

5%

Expert

2

17%

1

11%

Expert

3

25%

1

11%

TOTAL

15

4

TOTAL

17

3

 

Impact of exposure on representation change. We would expect a diversity of representations from novices, since they do not have much previous experience on which to form a representation. However, we might expect this diversity to decrease following substantial network use. With such a hypothesis, less useful conceptual representations would be discarded as experience increased. However, we found that, collectively, the diversity of conceptual representations of the Internet and the Web in fact increased with exposure to the Internet across all classes.

If we look at individual subjects and their responses over time, we can examine the ways in which their representations changed and, to some degree, how they reflect the emergent nature of network activity from their coursework. Table 5 shows all 24 of the subjects who responded to the Web representation question pre- and post-exposure, sorted by expertise. The representations are abbreviated in condensed statements as in Tables 2 and 3.

Table 6 shows an emergence of more descriptive conceptual representations, each in unique ways. There are four subjects (#2, #11, #35, and #44) whose representations over time reveal an enhanced articulation of a potentially "durable" representation as a result of network exposure. For example, the representations for subjects #2 and #35 take on a publishing component. This is an indication of their expertise gained in creating Web pages in their courses. Subject #11’s representation goes from a spatial library representation to one of an encyclopedia, which appears to graphically represent the information contained in the library. The conceptual representations for subject #44 stay aligned toward a brain representation.

Table 5
Response Differences Between Pre-and Post-Exposure Conceptual Representations (Sorted By Expertise)

Subject

Group

Expertise Factor

Pre-Exposure conceptual representation

Post-Exposure conceptual representation

2

TEP 231

novice

encyclopedia

information flow

10

EDPSY 387

novice

library

USA Today

11

CI 336

novice

library

encyclopedia - 24-hour

14

CI 336

novice

Web

Web with strands

21

EDPSY 387

intermediate

information

highway

25

CI 336

intermediate

BBS

international BBS

27

TEP 231

intermediate

Access to Internet

tangle of connections

31

TEP 231

intermediate

fungus

fungus

33

TEP 231

intermediate

streets

freeway

35

EDPSY 387

intermediate

information storerooms

publishing

36

EDPSY 387

intermediate

multimedia book

TV channel

38

TEP 231

intermediate

spider Web

library - floating pages

41

EDPSY 387

intermediate

spider Web

Web

42

EDPSY 387

intermediate

library

network operating system

43

EDPSY 387

intermediate

link - international

information sharing

44

EDPSY 387

intermediate

brain synapses

neural networks

45

EDPSY 387

intermediate

BBS

window on the world

46

TEP 231

intermediate

spider Web

spider Web

47

TEP 231

intermediate

[drawing]

[drawing]

52

EDPSY 387

expert

link - pictures

nebula - amorphous

53

TEP 231

expert

library - linked

supper table

57

TEP 231

expert

multimedia lines

connected computers

59

EDPSY 387

expert

library

city

61

TEP 231

expert

Web

community

 

Table 6
Actual Survey Responses for Subjects Whose Web Conceptual Representations Were Rated As Comparable

Subject

Pre-Exposure Response

Post-Exposure Response

2

Huge encyclopedia with unlimited information available to everybody

Information going back and forth in all directions. No beginning/no end [a drawing of a network of nodes with interconnecting arrows]

11

Sub-Libraries

24-hour encyclopedia

35

Multi-faceted, Individually Based Storerooms Of Information

Opportunity for Individuals To Immediately "Publish" + Utilize Information

44

something like the brain, with tons of synaptic connections

neural networks

 

While this analysis is exploratory, these data show that, for most subjects (17 out of 24), exposure to network activity, including publishing Web pages, resulted in a change in their expressed conceptual representations. Subjects’ previous life experiences, with or without respect to technology, probably have a significant impact on how they perceive the Internet. Furthermore, idiosyncratic experiences during the courses in which these subjects were enrolled must also have a significant impact. Finally, the changes we observed in this analysis appear to be unrelated to expertise.

Case Studies

The preceding discussion of the impact of exposure to representation change shows the difficulty of using survey statements alone to understand how conceptual representations emerge and are used by individuals. The following case studies reveal these relationships more deeply in the context of a problem-solving task.

Ten volunteers participated in an in-depth study of using the Web on two tasks: two (novices) from CI 336, four (intermediate) subjects from TEP 231, one (expert) from EDPSY 387 and three (experts) from within the UIUC College of Education (two graduate students and one faculty member). These last three experts were rated as such using the same survey instrument that was administered to the subjects in the courses. One task was to find resources on the Web relevant to constructing a lesson plan on a topic of the subject’s own choosing.

As in the survey study, we found diversity among the representations of these subjects. Initially we expected a difference in the kinds of conceptual representations used by experts compared to those used by novices. In our analyses of the case data, however, the findings can be seen most clearly by contrasting the case studies of the two novices with the case studies of the four experts.

Novices. Since the CI 336 class contained mostly practicing teachers with novice expertise levels, it was an appropriate and available group from which to solicit volunteers to participate in the case studies. The two students who volunteered were practicing K-12 teachers who had returned to UIUC for a master of education degree program. In the earlier questionnaire, Novice #1 described a "superhighway" representation of the Internet, and described the Web as "sort of like the Internet." Novice #2 described both the Internet and the Web as "kind of like a spider Web." This lack of specificity is not surprising, since both reported only having seen a demonstration of the Web during class the previous week. While the transcripts of their network problem solving efforts indicated relatively vague and undifferentiated representations, the case studies reveal emergent yet different views of the Internet in their teaching.

Novice #1. She elaborated on her representation of the Internet as "superhighway" with references to the national highway system. Her focus was on the use of the Internet as access to information, not so much on her conceptual representation of it:

"the Internet gets information from place to place easily, quickly and you have so much at your fingertips because of this network of everyone being connected and linked…I think that when I’m sitting down I’m trying to get somewhere. And, so, I’m trying to get information"

She talked further about her visions of using the Internet in her teaching and the effects on learning:

"You can’t just watch somebody do math problems all day or watch somebody run a science experiment and know what happened until you actually get your hands on [the problems]… That’s one of my initial things that I really want to find out is more [about] the interactive parts of the Internet because I think that will be more important to me rather than just having them sit down and finding these things and reading about them....I’m more interested in the interactive parts where you can actually do things and see things happen or ask questions."

This excerpt shows that Novice #1 possesses a distinct vision for what the Internet might mean to teaching and learning. However, at this point in her use of the Internet, she does not have a representation of it that guides how she uses it.

Novice #2. Her original representation of the Web was that it was a "series of connections to many sources." When we interviewed her, she elaborated:

"...It’s kind of like a spider Web. There’s kind of like a center court and there’re connections that go out and the connections come back to the center and they can go across or straight…we can go and connect with places all over the world…we can communicate to different people and they can communicate back..."

While she had not experienced the Web before this class, she did have some experience with concept mapping and semantic maps. She said that concept mapping formed the basis for her view of the Web:

"...[the Web] is [like] concept mapping where you have a broad topic and you branch out and those smaller subtopics get branched so I was kind of picturing that in my mind."

Novice #2 held representations that were relevant and plausible. For example, concept maps can be used to suggest the structure of the Web and semantic networks can suggest meaningful connections between nodes. However, in general, she had difficulty in applying potentially strong conceptions of the Web to her navigation of it.

With both of these novices, we see them struggling to comprehend a new medium for teaching and learning. Novice #1 had a vision of its use but lacked a substantive representation for how she might make better sense of it. Novice #2 showed how representations emerge over time and with experience. Her models made sense to her in other domains but she was unable to apply them to a new medium such as the Internet at that time. In both of these cases, each novice subject employed only a single representation.

Experts. In looking at the four case studies of people with considerable expertise with the Internet and the Web, we found their representations to be more fully articulated, especially in contrast to the novices. In their transcripts, three of the four experts described using at least two different representations. In fact, in two cases, the experts articulated reasons for using the different representations in different situations or how different representations were related to each other.

Expert #1. Here is an excerpt of one expert’s description of his conceptual representations and how he employed them:

"The overwhelming conceptual representation I have for the Internet is what is called the rhizome...[the Web is] also pretty rhizomatic. Rhizome is contrasted with the tap root representation ...central tap root, [describing his search strategy] but I think although theoretically, I think that rhizomatic structures are interesting I still think....tap root hierarchical fashion. Let’s go to the central sources first - let’s get those names things and then… "

The rhizome representation described by this expert refers, biologically, to grasses, mushrooms, and similar plants that consist of an underground interconnected root system with surface extensions (nodes) that emerge from the root system . This was an emergent (at the time of this case study) metaphorical conception of the Internet for this expert that conveyed a much more decentralized view of how information is constructed and presented on the Web and who constructs it. (See Brown and Deleuse & Guattari for more in-depth treatments of this metaphor and its relationship to the Web.) This expert’s representation reveals his two opposing but plausible conceptual representations of the Internet, which can be employed in different ways and for different purposes. His rhizome representation was adopted from ideas he had heard in correspondence with other educators. Yet, since he was a philosopher, they fit his existing beliefs about information access and ownership that could be described as decentralized yet interconnected, much like the rhizome structure. However, it was clear from this excerpt that while his beliefs about the structure of the Internet are broadly rhizomatic and non-hierarchical, he operationalized his searching strategies using the tap-root or hierarchical representation. The subject held these two representations simultaneously yet he differentiated what their purposes were given a task of searching and navigating the Web. It is also evident in this excerpt that the subject was still negotiating within himself what his representations were and what to do with them.

Expert #2. The second expert explained his conceptual representations of the Internet and the Web by describing how it evolved for him over time. He talked about how his experience with the Internet starting with using e-mail as a way to transfer text files to another individual: "…I had that vision of kind of a two-way stream of information being able to go back and forth." He then said that this representation caused him problems when he tried to explain the Internet to others: "…I had a very difficult time because I think I had a single notion of sending files by a vehicle like e-mail." He reported the transition: "…my model changed from this back and forth file transfer by e-mail to being a sort of multiple parallel root thing." In using the Web in his task, he, like Expert #1, described two contrasting representations. "…The usual model that I had in mind for the Web I think was the one I was given by the name of it." This led him to a search strategy that assumed there was "…a place I want to be and I just need to find my way there." However, he also expressed a representation that he said had evolved over experience, a "salvage yard model … a place where there is a lot of stuff there that I don’t want and if I need a part of my car I’m going to go to the Ford section and there might be a light working on one and a clutch cable on another and I just kind of strip from it what I want." One concrete implication of this "salvage yard" representation that he described is that "…I’m more inclined to scroll than I am with the Web model." He extended this representation to apply to e-mail and listservs as well.

Expert #3. The third expert expressed a representation of the Internet as "…a computer network -- all the computers are connected. And, you have information on this computer, you have information on that computer and then you, any computer that is on the network you can access information anywhere." This fairly concrete representation then shifted to a more abstract representation: "…People are doing is to organize the information…categorize so that it becomes easier to, people to access. So, in thinking of development I always think that maybe later libraries will disappear in the end. All the books -- if you publish a book you have -- I’m sure they’ll have the electronic form somewhere and that will be easy to … put on the Web. So then that book will be there and anyone can access so, searching for a book will become easier." He expressed two representations, a network of computers representation and a "place to look for" representation.

Expert #4. The fourth expert said that she didn’t explicitly use a representation in her use of the Internet or the Web: "I don’t think there’s any relation at this point — maybe when I was starting out. But at this point I’ve got a pretty good concept of what’s there that I don’t think about it anymore. Maybe subconsciously I do, but I don’t think about it really." However, she described several representations that she uses in training others to use the Internet and the Web. One representation was a "city" representation: "Well, if I went with the city buildings, the city model, I could explain to them that yeah, you look in a phone book, but sometimes when you are already there you’re just going to walk from store to store, you’re going to walk from place to place looking for what you want to find." She described that this "city" representation helped her in her searching process: "And you can make some guesses just like if you see that on one block there doesn’t happen to be a restaurant you, you know that the next block — there’s going to be a restaurant because just out of experience you know these things are spaced in certain ways." But she also described several other representations she’s used in training: "…I’ve used the spider Web example, the post office example…another one that I use to explain is I just use the phone company, how we are all connected." She described her development as starting at a young age "I was brought up pretty much on this stuff already being there…the computer was always there. So, it was just like a book is always there. Yes, you learn how to read but you don’t necessarily think about what you go through to read words anymore. It’s just natural." Expert #4 was the youngest of the four experts, and had started working with computers at a much younger age than the other three.

In these case studies, two phenomena stand out:


Discussion

The findings in this study are similar to the outcomes of previous research that has investigated expert and novice problem solving in other domains . For example, novice physics problem solvers have been found to think about physics problems in a uni-dimensional manner (e.g., they start writing equations), while expert physics problem solvers have multiple ways that they select among and switch between in the process of problem solving. Related ethnographic research by Kozma and associates revealed that practicing chemists utilized a variety of representations consisting of symbols and signs (e.g., structural diagrams of compounds, NMR spectra of experimental results) as part of a dynamic process of generating hypotheses, confirmation or disconfirmation of hypotheses, collaborative interpretation of the meaning of the representations, and so on. Other research by Kozma and Russell found significant differences between expert and novice chemists in terms of their ability to use multiple representational forms to construct an understanding of chemistry. Research in other domains, such as climatology and biology has also documented the flexible nature of representational expertise.

Earlier, we asked whether expert conceptions of technologies such as the Internet and the Web were different from novice conceptions. The types of expert and novice conceptions we found were not qualitatively different, although they did vary in the degree of elaboration and detail. However, the important differences between experts and novices that were identified in this research relate to the utilization of these conceptions. Specifically, experts were found to flexibly use a variety of conceptual representations -- sequentially, in parallel, or in more complex ways--depending on the individual and the task. Novices use a single conceptual representation.

This leads us to a view of "expertise" that is not just the acquisition of a "truer" or "better" representation than what a novice might have. Rather, we view experts as having a larger "toolkit" of representations for a domain of expertise (e.g., Internet and Web technologies), and having the knowledge of which representations to select for particular tasks. A master carpenter has a variety of tools that she knows how to use and also knows which to use for which purpose. A novice carpenter might know only how to use one basic tool, and might try to use that tool for all purposes. However, over time, the novice does not give up the basic tool, but instead learns to use additional specialized tools.

Similarly, the novice’s initial conceptual representation is not necessarily something to be taken away, to be replaced by the "right" representation. Instead, the novice needs to learn multiple ways to think about the domain, and to learn when and in what sequence to use the conceptual representations in their "representational toolkit" in order to accomplish tasks in the domain of expertise.

To carry the analogy of the representational toolkit further, the research on novice-expert differences points to the use by experts of qualitative representations early on in the problem solving process, and the value of those qualitative representations in guiding the application of more detailed specific representations . This is similar to the carpenter's selection and use of "coarse-grained" tools, to roughly shape a designed object, and then later more "fine-grained" tools to finish the construction of the object. The expert carpenter selects a sequence of tools to accomplish a task; an expert in a domain selects a sequence of conceptual representations to accomplish a cognitive task.

An expert also knows how to switch representations when he or she is blocked in problem solving (or, in the case of Expert #4, in teaching). Each representation is effective for accomplishing some cognitive tasks and less effective for others. An expert who runs into a problem can switch representations, which will often allow him or her to see the way around the cognitive block. A novice when blocked has no alternative representations to switch to.

The experts that we have studied here have many ways to think about the Internet and Web and they use metacognitive strategies to choose which representation to use for a given task. Again, there may not be a "right" or "best" representation of the Internet or the Web, but rather a multitude of representations, each of which is helpful for guiding a person in certain tasks, not helpful in other tasks, and harmful in accomplishing yet others.

Our research suggests that the first key element of a representational toolkit is acquiring multiple representations. A second key is learning to coordinate these multiple representations and to switch from one to another as needed. Finally, a third key is to learn which representations are good for which purposes (and vice versa) so that one can select a sequence of representations that accomplishes the task at hand and allows the expert to overcome any problems encountered along the way.

Conclusion

We asked two questions at the onset of these studies: (a) Does a teacher’s conceptual representation of a technology have an impact on how he or she uses it? and (b) Do experts think about technologies such as the Internet and the World Wide Web in ways different than novices? The survey data and the case studies, considered together, provide partial answers to both questions. There are four central findings from these studies.

These data, in general, reveal the complexity of obtaining and describing conceptual representations in an ill-defined domain of knowledge and action. Also, we found it generally easier and more productive for a person to express his or her conceptual representation when engaged in a task. There are many representations that are, in some way, plausible when compared with technical descriptions of the Internet (not that technical descriptions are the goal of these studies, but they provide a common, albeit limited, language for comparison). The fact that the subject expressed a representation at all, especially one that was unique, is as important as how the representation was employed. From a cognitive perspective, these are two emergent dimensions arising from this series of studies. While our survey data express the many unique representations and the case studies reveal how the representations are employed, these studies raise more questions than they answer. Future analyses of representational change at the individual level may lead us to understand the relationship between an initial representation and its maintenance, adaptation, or elimination over time.

The ability of the Internet and its most common applications to support a diversity of views may be to a considerable extent the basis of the "ease of use" that people report. As Internet applications become more heavily graphic in nature, we may see the unexpected result that these applications become easier to use for some people (those who hold and use conceptual representations that contain the same visual elements), but actually harder for many others to use. Instead, a solution may be to provide a diversity of interfaces and easier ways to switch among those different interfaces. In this way, multiple conceptual representations in the "representational toolkit" framework may be useful to the developers of network learning environments.

For teachers, technology can be considered a new form of expertise that is being imposed upon, or at least being integrated into, the practice of teaching. Teachers, at all levels in their careers, are encountering the demands of technological expertise in unique ways. This is the challenge for teacher preparation programs and for in-service professional development. When preparing teachers to be active consumers and producers of knowledge on the Internet, it is more important to examine the epistemic nature of their stated representation than the accuracy or "portability" of any representation to training. The subjects in these studies often used metaphors to express an underlying belief system about the Internet or technology in general. We have examined some of the conceptual representations that a few teachers have of the Internet, but what are teachers’ beliefs about the Internet? Certainly, it is not meaningful to seek a generalized answer to this question. But, while these data uncover some of those beliefs, our experience with teachers using the Internet has shown that their beliefs about technology in their teaching practices have formed over many years and are quite complex. To better support teachers we should engage them in a more critical exploration of these technologies that will have a lasting impact on their personal understanding of it, a process by which conceptual representations are but one construct for understanding. The selection of any one conceptual representation as a framework for training may be helpful to some teachers but not at all helpful to others. The goal of technology training for teachers should be to help teachers develop multiple coordinated conceptual representations that they can use at appropriate times to help them achieve their goals for using educational networks. A "representational toolkit" is one framework by which we can understand and support teachers' emergent expertise with technology in relation to their teaching practice.

Finally, a "representational toolkit" allows us to conceptualize expertise in a much more developmental perspective. The conceptual representation of a novice is a beginning tool, not necessarily something to be taken away from the novice but instead the first tool of many that the learner will add to his or her "representational toolkit." As each tool is added, the learner needs to know not only how to use the tool, but also when to use the tool, and how to switch from one representational tool to another as the nature of the task at hand changes. Some tools will be more useful in the initial stages of a task in the domain, while others are typically more useful in the later stages. An expert is a master of his or her tools, which means both knowledge about the tool and the metacognitive knowledge about selecting the sequence of tools used. This view of expertise encourages teachers to build upon the diverse set of conceptual representations that novices start with, rather than ignoring or actively quashing them. By building on the naturally occurring diversity of novice conceptual tools, we can help teachers to develop expertise with a rich set of representational tools that they can use to meet the challenges of our increasingly dynamic and diverse world.


References

Brown, P. (1997). The rhizome. [On-line]. Available: http://communication.students.rmit.edu.au/1996/Pennie_Brown/Rhizome/rhizome.html.

Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5, 121-152.

Chi, M. T. H., Glaser, R., & Farr, M. J. (Eds.). (1988). The nature of expertise. Hillsdale, NJ: Lawrence Erlbaum.

Deleuse, G., & Guattari, F. (1987). A thousand plateaus: Capitalism and schizophrenia (B. Massumi, Trans.). Minneapolis: University of Minnesota Press.

Ericsson, K., & Simon, H. (1984). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press.

Garner, R., & Gillingham, M. (1996). Internet communication in six classrooms: Conversations across time, space, and culture. Mahway: NJ: Lawrence Erlbaum.

Gentner, D., & Gentner, D. R. (1983). Flowing waters or teeming crowds: Mental models of electricity. In D. Gentner & A. L. Stevens (Eds.), Mental models. Hillsdale, NJ: Erlbaum.

Honey, M., & Henríquez, A. (1993). Telecommunications and K-12 educators: Findings from a national survey (Technical Report). Bank Street College of Education.

Johnson-Laird, P. N. (1983). Conceptual representations: Towards a cognitive science of language, inference, and consciousness. Cambridge, MA: Harvard University Press.

Kindfield, A. C. H. (1994). Biology diagrams: Tools to think with. Journal of the Learning Sciences, 3(1), 1-36.

Kozma, R., Chin, E., Russell, J., & Marx, N. (1997). The roles of representations and tools in the chemistry laboratory (Technical Report). Menlo Park, CA: SRI International.

Larkin, J. H., McDermott, J., Simon, D. P., & Simon, H. A. (1980). Expert and novice performance in solving physics problems. Science, 208, 1335-1342.

Lowe, R. (1987). Drawing out ideas: A neglected role for scientific diagrams. Research in Science Education, 17, 56-66.

McCloskey, M. (1983). Naive theories of motion. In D. Gentner & A. L. Stevens (Eds.), Mental models. Hillsdale, NJ: Lawrence Erlbaum.

Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall.

Norman, D. A. (1983). Some observations on mental models. In D. Gentner & A. L. Stevens (Eds.), Mental models. (pp. 7-14) Hillsdale, NJ: Lawrence Erlbaum.

Norman, D. A. (1988). The design of everyday things. Reading, MA: Addison-Wesley.

Norman, D. A., & Draper, S. W. (Eds.). (1986). User centered system design: New perspectives on human-computer interaction. Hillsdale, NJ: Lawrence Erlbaum.

Thomas, L. (1997). Self as teacher mediated by computer technologies in learning to teach. Unpublished Unpublished doctoral dissertation, University of Illinois, Urbana-Champaign, Champaign, IL.

 


Footnotes

* A version of this paper was presented at the AERA Annual Meeting, New York, April 10, 1996. Portions of this research were supported by the National Science Foundation under Grant No. RED-9253423. The Government has certain rights in this material. Any opinions, finding, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

1Building from the literature, we had used the term "mental model" during our data collection. We have subsequently adopted "conceptual representation" as a more general term.

 


Appendix A

Network Mental Models Questionnaire1

We are conducting a study of how best to teach people about computer and information networks. We’d like you to fill out the questions below. We will protect your identity in any reports of our findings. If you agree to participate, please sign below.

Signature: Date:

Please contact me if you have any questions. Thanks!

Jim Levin 244-0537 Room 130 Education Building

Please circle the appropriate choice below:

My experience of teaching: none 1-5 years 6-10 years 11-20 21 or more

My previous experience with computers:none little some quite a bit lots

My previous experience with networks: none little some quite a bit lots

Previous experience with e-mail:none little some quite a bit lots

Previous experience with newsgroups:none little some quite a bit lots

Previous experience with Gopher:none little some quite a bit lots

Previous experience with WWW:none little some quite a bit lots

Other network experiences:

We are interested in what "mental models" you have for using computer networks. For example, people often think about the atom as being like a miniature solar system. The solar system is then a mental model of the atom.

What "mental model(s)" do you use to think about the Internet?

What "mental model(s)" do you use to think about electronic mail?

What "mental model (s)" do you use to think about the World-Wide Web?

Would you be willing to talk with us in more detail about your mental models?

Yes No

If yes, please write your phone number here:

What days and times would you be available this week to meet for about 30 minutes?


Appendix B

NETWORK MENTAL MODELS STUDY1

Experimenter Instructions for Think Alouds

Read the following instructions to the subjects:

In this study we are interested in what you think to yourself as you perform some tasks that we give you. In order to do this, we will ask you to think aloud while you are doing the network tasks on the computer. What I mean by think aloud is that I want you to say out loud everything that you would ordinarily think to yourself silently. Just act as if you are alone in the room speaking to yourself. If you are silent for any length of time I will remind you to keep thinking aloud. At some points during the session, I might ask you some questions about what you are doing.

Any questions?

Before beginning, why don’t we start with a practice problem for thinking aloud. I want you to think aloud while you solve this problem. I want you to multiply two numbers in your head.

Think aloud while you multiply 24 times 34.

AFTER THEY ARE DONE, ASK THEM HOW IT WAS. THEN ASK IF THEY WOULD YOU LIKE TO PRACTICE ANOTHER THINK ALOUD. IF SO, THEN:

Think aloud while you tell me how many windows there are in your house.

ID NUMBER ______________________

EDUCATIONAL TASKS FOR USING NETWORK

Network Task 1: School Computer Consultant

Instructions: Read the problem scenario below and then use the World Wide Web to find resources addressing the various issues in the scenario. You will have a total of 20 minutes to find relevant materials on the Web. Add the documents you feel are relevant to the "Bookmark" list.

We also would like you to "think aloud" as you do these tasks. Just say aloud the ideas that come to you as you work on the Web and make your selections. Do you have any questions?

You are an educational technology consultant to the school board of a large urban school district. The board has been very concerned over the declining quality of education both at the national level in the United States and in the district itself.

There is a split between members of the board over the use of expensive educational computer technologies to improve the quality of education in the district. Some members of the board favor the extensive use of educational technologies. They note that educational research has found the use of media and computers to be a valuable instructional approach in many areas such as mathematics, science, and writing. Furthermore, they argue students must be prepared to use computers that are becoming increasingly common in society. These board members also argue minority students, in particular, must receive exposure to computers to insure their ability to successfully compete in higher education and in the work place.

Other board members feel that spending money for computer technology will reduce funds available for teacher salaries (which are already below the national average), enrichment programs such as music and art, and library acquisitions. It is more important, they argue, for schools to concentrate on teaching the students to think. Colleges and businesses--who have greater financial resources--should then be required to provide the training computer skills that college students and employees must use.

Use the World Wide Web to find documents that (a) analyze key issues related to the impact of educational technologies on schools, and (b) suggest a resolution for the conflict between the two factions on the school board. Feel free to write down any notes or thoughts you have on the back side of this page.

Network Task 2: Lesson Plan with Network Resources

Instructions: Imagine that you have a class of your choice to teach tomorrow and that you plan to use resources for the class that were derived from the "net." You will have a total of 20 minutes to find relevant materials on the Web. Add the network documents you feel are relevant to the "Bookmark" list. Also, please tell us:

As with the first task, we want like you to "think aloud" as you do this task. Just say aloud the ideas that come to you as you work on the Web and make your selections. Do you have any questions?

Feel free to write down any notes or thoughts you have on the front and back of this page.


Last updated: 2 Dec 1999