Mayer et al. (2016). Development of novices’ professional knowledge networks within the contexts of classroom teaching and information searches on the internet. Research data from a longitudinal study 2013-2015.

Bibliographic Information

Creator: Mayer, Anne-Kathrin; Rosman, Tom; Birke, Peter; Gorges, Johannes; Krampen, Günter

Contributor: Mayer, Anne-Kathrin; Rosman, Tom; Birke, Peter; Gorges, Johannes; Krampen, Günter

Funding: Joint Initiative for Research and Innovation; Leibniz Competition (SAW 2013)

Title: Development of novices’ professional knowledge networks within the contexts of classroom teaching and information searches on the internet. Research data from a longitudinal study 2013-2015

Year of Publication: 2016

Citation: Mayer, A.-K., Rosman, T., Birke, P., Gorges, J., & Krampen, G. (2016). Development of novices’ professional knowledge networks within the contexts of classroom teaching and information searches on the internet. Research data from a longitudinal study 2013-2015 [Translated Title] (Version 1.0.0) [Data and Documentation]. Trier: Center for Research Data in Psychology: PsychData of the Leibniz Institute for Psychology ZPID. https://doi.org/10.5160/psychdata.mrae15ent24

Abstract

The study aims at describing and analyzing the development of professional knowledge networks in psychology and computer science students (first to fourth semester). The study primarily focuses processes of restructuring knowledge (conceptual change) following the transition from secondary education (school) to tertiary education (university). Three domains of knowledge are investigated: (1) domain-specific knowledge, (2) information literacy, and (3) epistemic beliefs.

The study employs a quantitative four-wave longitudinal design. To gain empirical data on knowledge development, both psychology (N = 137 at the first wave) and computer science students (N = 89 at the first wave) were investigated by means of standardized tests. The first wave took part right at the beginning of students’ first semester, followed by three consecutive waves at the beginning of the second, third, and fourth semesters. Additionally, data on several covariates likely to influence knowledge development (e. g., cognitive ability, academic self-concept, learning and achievement motivation) were collected.

Codebook

Study Description

Research Questions/Hypotheses:

Within the project, three research guiding questions were addressed:

  1. development of methods to capture knowledge networks in the disciplines of psychology and computer science.
  2. descriptive analysis of changes in domain-specific knowledge, information literacy, and epistemological beliefs over the first three semesters and explanation of these changes by cognitive and motivational variables
  3. exploration of ways to promote domain-specific knowledge, information literacy, and epistemological beliefs in students.

Research Design: Partially Standardized Survey Instrument (provides question formulation; open answer format); repeated measurements

Measurement Instruments/Apparatus:

The individual measurement time points (t1, t2, t3 and t4) each comprised two data collections: First, the subjects had to complete an online questionnaire battery (approx. 1/2 to 1 hour) in a setting of their choice (e.g., at home). Subsequently, a test battery with the various performance tests had to be completed in the context of approx. 2-hour standardized group surveys in PC pools of the respective university (Trier University, Trier University of Applied Sciences, Saarland University).

Central procedures (recorded at all 4 measurement time points [t1-t4]):
Specialized knowledge in psychology: Here, a specially developed procedure was used to record specialized knowledge in memory psychology. The test measures the extent to which students understand that information is not stored statically in human memory, but is processed and changed by processes such as interference, chunking, and source control. For this purpose, subjects are given k = 9 descriptions of classical memory experiments. For each of the nine situation descriptions, subjects are given six possible outcomes of the experiment with different rationales. Subjects indicate on a seven-point rating scale how much they believe that each described outcome of the experiment and its rationale are correct or incorrect. Only one of the six statements in each case describes the actual outcome of the experiment with the explanation accepted in the research literature. Details on test construction and evaluation can be found in Gorges, Schneider, and Mayer (2015).

Expertise in computer science: The specially developed procedure for assessing algorithmic understanding measures the extent to which students can evaluate algorithmic strategies for the systematic solution of “classical” problems from computer science. The problem definitions or solution strategies are, on the one hand, the two common strategies “divide and conquer” and the so-called “greedy strategy”. On the other hand, the ability to recognize when probably no efficient strategy is helpful to solve the problem exactly (“NP-completeness”), as long as P = not NP. The short form of the procedure comprises k = 6, the long form k = 9 exemplary descriptions of classical problems, which are equally distributed among the three strategies, i.e. two (respectively three) descriptions per strategy. For evaluation, a seven-point rating scale is given in each case.
In the problems to be solved using the “divide and conquer” strategy, the subjects are given six algorithms that are to be evaluated for their efficiency with respect to their runtime using O-notation. The algorithms presented can be assigned to three efficiency levels: Two each solve the problem efficiently, two very inefficiently, and the remaining two have a runtime that can still be considered acceptable. For the descriptions of the “Greedy Strategy”, six solution proposals are given together with the problem definition, which are to be evaluated with respect to the correctness of the procedure; in each case, only two of the six statements are proven to be correct in the technical literature. In the situation descriptions for “NP-completeness”, six heuristics each are given for solving the problem, and the subjects are also asked to rate these for correctness on a seven-point rating scale. Again, only two of the six heuristics represent statements that are assumed to be correct in the literature as long as P = not NP.

Information Literacy: The so-called PIKE test was constructed according to the principle of the so-called Situational Judgement Tests: The items each describe a problem situation in the research process in one or two sentences and then list four possible procedures that are to be assessed in terms of their instrumental suitability for solving the problem on a scale from 1 = not at all suitable to 5 = very suitable. Both unsuitable and in principle suitable, but differently good or efficient solutions are given. Due to different research cultures and approaches in psychology and computer science, the test was adapted to each discipline (PIKE-P for psychology and PIKE-CS for computer science). Details on test construction and scoring can be found in Rosman and Birke (2015) and Rosman, Mayer, and Krampen (2015).

Epistemological beliefs: Epistemological beliefs were assessed using two different procedures, each discipline-specific. The EBI-AM questionnaire is based on established questionnaires on epistemological beliefs. Compared to these, however, it has the advantage with regard to the questions of the present study of representing absolute and multiplistic beliefs on separate scales. The instrument contains 23 epistemological statements; subjects are asked to rate the degree of their agreement with reference to their respective scientific discipline (psychology or computer science) on a five-point Likert scale (e.g., “The only certain thing in this discipline seems to me to be uncertainty”). Details on test construction and scoring can be found in Peter, Rosman, Mayer, Leichner, and Krampen (2015).
In addition, the established CAEB was used. On a semantic differential, subjects are asked to assess knowledge in their discipline (psychology or computer science). The instrument maps the dimensions of texture and variability of knowledge. Details on test construction and evaluation can be found in Stahl and Bromme (2007).

In addition, a large number of covariates were recorded at different survey time points, including epistemic curiosity, essentialism, intelligence, tendency to drop out, study satisfaction, certainty of study choice, personality, self-concept, and others.

Data Collection Method:

Data collection in the absence of an experimenter (Online questionnaire):
– Individual Administration
– Computer-Supported

Data collection in the presence of an experimenter
– Group Administration (2-30 subjects per group)
– Computer-Supported
– Paper and Pencil (only Raven’s, APM intelligence test)
– Individual Administration (only exceptional cases)

Population: College students (psychology and computer science)

Survey Time Period:

Psychology:
1st wave (baseline): october – november 2013
2nd wave: march -mai 2014
3rd wave october-november 2014
4th wave: march-mai 2015

Computer science:
1st wave (baseline): october 2013 – january 2014
2nd wave: april -june 2014
3rd wave october-november 2014
4th wave: april-mai 2015

Sample: Convenience sample

Gender Distribution:

Psychology (t1):
82% female subjects
18% male subjects

Computer science (t1):
22% female subjects
78% male subjects


Age Distribution: Psychology: 18-31 years; computer science: 17-32 years

Spatial Coverage (Country/Region/City): Germany/Rhineland-Palatinate & Saarland/Trier & Saarbrücken

Subject Recruitment: Recruitment was done through emails, flyers, notices on each campus, and advertising in classes. Participants received an expense allowance at the end of each measurement time point. Subjects who participated in all four measurement time points were paid a bonus after completion of the fourth measurement time point.

Sample Size: Psychology: 137 inidividuals (t1); Computer science: 89 indiviudals (t1)

Return/DropOut:

Psychology: approximately 70-80% exhaustion rate and 84% survival rate from t1 to t4 (t2: 126 individuals; t3: 116 individuals; t4: 115 individuals).
Computer science: about 20-30 % exhaustion rate and 64 % survival rate from t1 to t4 (t2: 68 individuals; t3: 62 individuals; t4: 57 individuals)

Literature

Publications Directly Related to the Dataset
Publications Directly Related to the Dataset
Birke, P., Rosman, T. & Mayer, A.-K. (2016). Entwicklung fachspezifischer epistemologischer Überzeugungen bei Studienanfängern der Psychologie und Informatik. In A.-K. Mayer & T. Rosman (Hrsg.), Denken über Wissen und Wissenschaft: Epistemologische Überzeugungen als Gegenstand psychologischer Forschung. Lengerich, Germany: Pabst Science Publishers.
Datensatz 0314216
Birke, P., Rosman, T., Mayer, A. K., & Walter, B. (2014). A domain-specific test of procedural knowledge about information searching for students of computer science. In S. K. S. Špiranec, E. G. D. Mizrachi, & R. Catts (Eds.), Information literacy. Lifelong learning and digital citizenship in the 21st century (pp. 683-692). Cham, Switzerland: Springer International Publishing.

Datensatz 0306896
Peter, J., Rosman, T., Mayer, A.-K., Leichner, N. & Krampen, G. (2015). Assessing epistemic sophistication by considering domain-specific absolute and multiplicistic beliefs separately. British Journal of
Educational Psychology. Advance online publication.Datensatz 0307202
Rosman, T., Mayer, A.-K., Kerwer, M. & Krampen, G. (conditionally accepted). The differential development of epistemic beliefs in psychology versus computer science students: A four-wave longitudinal study. Leibniz-Zentrum für Psychologische Information und Dokumentation, Trier.
Rosman, T., Mayer, A.-K. & Krampen, G. (2015). Intelligence, academic self-concept, and information literacy: The role of adequate perceptions of academic ability in the acquisition of knowledge about information searching. Information Research, 20(1), Special supplement: Proceedings of ISIC: The Information Behaviour Conference, Leeds, United Kingdom, paper isic34.Datensatz 0297274
Rosman, T., Mayer, A.-K. & Krampen, G. (2016). On the pitfalls of bibliographic database searching: Comparing successful and less successful users. Behaviour and Information Technology, 35 (2), 106-117.Datensatz 0297273
Utilized Test Methods
Utilized Test Methods
Ahn, W., Flanagan, E. H., Marsh, J. K. & Sanislow, C. A. (2006). Beliefs about essences and the reality of mental disorders. Psychological Science, 17(9), 759-766.
Behm, T. (2015). Informationskompetenz und Selbstregulation: Zur Relevanz bereichsspezifischer Selbstwirksamkeitsüberzeugungen. In A.-K. Mayer (Hrsg.), Informationskompetenz im Hochschulkontext – Interdisziplinäre Forschungsperspektiven (pp. 151-162). Lengerich, Germany: Pabst Science Publishers.Datensatz 0286015
Birke, P., Rosman, T., Mayer, A. K. & Walter, B. (2014). A domain-specific test of procedural knowledge about information searching for students of computer science. In S. K. S. Špiranec, E. G. D. Mizrachi, & R. Catts (Eds.), Information literacy. Lifelong learning and digital citizenship in the 21st century (pp. 683-692). Cham, Switzerland: Springer International Publishing.Datensatz 0306896
Dickhäuser, O., Schöne, C., Spinath, B. & Stiensmeier-Pelster, J. (2002). Die Skalen zum akademischen Selbstkonzept: Konstruktion und Überprüfung eines neuen Instrumentes. Zeitschrift für Differentielle und Diagnostische Psychologie, 23, 393-405.Datensatz 0158819
Fellenberg, F. & Hannover, B. (2006). Kaum begonnen, schon zerronnen? Psychologische Ursachenfaktoren für die Neigung von Studienanfängern, das Studium abzubrechen oder das Fach zu wechseln. Empirische Pädagogik, 20(4), 381-399.Datensatz 0196676
Gorges, J. F., Schneider, M., & Mayer, A.-K. (2015, August). A latent transition analysis of psychology students' developing understanding of human memory. 16th Biennial EARLI Conference for Research on Learning and Instruction, Limassol, Cyprus.
Haslam, N., Bastian, B. & Bissett, M. (2004). Essentialist beliefs about personality and their implications. Personality and Social Psychology Bulletin, 30(12), 1661-1673.Datensatz 1553624
Kalish, C. W. (2002). Essentialist to some degree: Beliefs about the structure of natural kind categories. Memory & Cognition, 30(3), 340-352.
Körner, A., Geyer, M., Roth, M., Drapeau, M., Schmutzer, G., Albani, C., Schumann, S., & Brähler, E. (2008). Persönlichkeitsdiagnostik mit dem NEO-Fünf-Faktoren-Inventar: Die 30-Item-Kurzversion (NEO-FFI-30). Psychotherapie, Psychosomatik, Medizinische Psychologie, 58, 238-245.Datensatz 0208749
Krauss, S., Kunter, M., Brunner, M., Baumert, J., Blum, W., Neubrand, M. et al. (2004). COACTIV: Professionswissen von Lehrkräften, kognitiv aktivierender Mathematikunterricht und die Entwicklung von mathematischer Kompetenz In J. Doll & M. Prenzel (Hrsg.), Die Bildungsqualität von Schule: Lehrerprofessionalisierung, Unterrichtsentwicklung und Schülerförderung als Strategien der Qualitätsverbesserung (S. 31–53). Münster: WaxmannDatensatz 0184107
Leichner, N., Peter, J., Mayer, A. K. & Krampen, G. (2014). Assessing information literacy programmes using information search tasks. Journal of Information Literacy, 8(1), 3-20.Datensatz 0280998
Liepmann, D., Beauducel, A., Brocke, B. & Amthauer, R. (2007). Intelligenz-Struktur-Test 2000 R (I-S-T 2000 R). Göttingen: Hogrefe.
Lister, R., Simon, B., Thompson, E., Whalley, J. L. & Prasad, C. (2006). Not seeing the forest for the trees: Novice programmers and the SOLO taxonomy. ACM SIGCSE Bulletin, 38(3), 118-122.
Litman, J. A. & Mussel, P. (2013). Validity of the interest-and deprivation-type epistemic curiosity model in Germany. Journal of Individual Differences, 34(2), 59-68.Datensatz 0270163
Mayer, A.-K. & Friebe, J. (2015, September). Gesundheitsbezogenes Informationsverhalten älterer Erwachsener – ein Beitrag zu gesundem Altern? 12. Kongress der Fachgruppe Gesundheitspsychologie, Graz, Österreich.
Mayer, A.-K. & Krampen, G. (2015, Juli). The »Scale Openness for Information« (SOFI) – A new assessment tool for research on information behavior. Paper presented at 13th Conference on Psychological Assessment (ECPA), Zurich, Schweiz.
Niedzwienska, A., Neckar, J. & Baran, B. (2007). Development and validation of the Implicit Memory Theory Scale. European Journal of Psychological Assessment, 23(3), 185-192
Peter, J., Leichner, N., Mayer, A.-K. & Krampen, G. (2015a). A short test for the assessment of basic knowledge in psychology. Psychology Learning & Teaching, 14(3), 224–235.Datensatz 1083623
Peter, J., Rosman, T., Mayer, A.-K., Leichner, N. & Krampen, G. (2015b). Assessing epistemic sophistication by considering domain-specific absolute and multiplicistic beliefs separately. British Journal of Educational Psychology. Advance online publication. doi: 10.1111/bjep.12098Datensatz 0307202
Raven, J., Raven, J.C., & Court, J.H. (1998). Raven manual section 4: Advanced Progressive Matrices. Oxford: Oxford Psychologists Press.
Rosman, T., Mayer, A.-K. & Krampen, G. (2015). Measuring psychology students’ information-seeking skills in a situational judgment test format: Construction and validation of the PIKE-P Test. European Journal of Psychological Assessment. Advance online publication.Datensatz 0291381
Schlink, S., & Walther, E. (2007). Kurz und gut: Eine deutsche Kurzskala zur Erfassung des Bedürfnisses nach kognitiver Geschlossenheit. Zeitschrift für Sozialpsychologie, 38(3), 153-161.Datensatz 0200685
Schöne, C., Dickhäuser, O., Spinath, B. & Stiensmeier-Pelster, J. (2002). Die Skalen zur Erfassung des schulischen Selbstkonzepts (SESSKO). Göttingen: Hogrefe.Datensatz 9004611
Shtulman, A. & Valcarcel, J. (2012). Scientific knowledge suppresses but does not supplant earlier intuitions. Cognition, 124(2), 209-215.
Spies, K., Westermann, R., Heise, E. & Schiffler, A. (1996). Diskrepanzen zwischen Bedürfnissen und Angeboten im Studium und ihre Beziehungen zur Studienzufriedenheit. Empirische Pädagogik, 10, 377-409.Datensatz 0111173
Spinath, B. (1998). Implizite Theorien über die Veränderbarkeit von Intelligenz und Begabung als Bedingungen von Motivation und Leistung. Dissertation, Universität Bielefeld, Fakultät für Psychologie und Sportwissenschaft.Datensatz 0133464
Stahl, E. & Bromme, R. (2007). The CAEB: An instrument for measuring connotative aspects of epistemological beliefs. Learning and Instruction, 17(6), 773-785.Datensatz 0204849
Venkatesh, V., Morris, M. G., Davis, G. B. & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425-478.
Wilhelm, O., Hildebrandt, A. & Oberauer, K. (2013). What is working memory capacity, and how can we measure it. Frontiers in Psychology, 4, paper 433.Datensatz 0276477
Rosman, T., & Birke, P. (2015). Fachspezifische Erfassung von Recherchekompetenz durch prozedurale Wissenstests: Psychologie vs. Informatik. In A.-K. Mayer (Hrsg.), Informationskompetenz im Hochschulkontext – Interdisziplinäre Forschungsperspektiven (S. 103-120). Lengerich: Pabst Science Publishers. Datensatz 0286013
Further Reading
Further Reading
Hofer, B. K., & Pintrich, P. R. (1997). The development of epistemological theories: Beliefs about knowledge and knowing and their relation to learning. Review of Educational Research, 67(1), 88-140.
Rosman, T., Mayer, A.-K. & Krampen, G. (2015). Combining self-assessments and achievement tests in information literacy assessment: Empirical results and recommendations for practice. Assessment and Evaluation in Higher Education, 40(5), 740-754. Datensatz 1064743
Schneider, M. & Hardy, I. (2013). Profiles of inconsistent knowledge in children's pathways of conceptual change. Developmental Psychology, 49(9), 1639-1649.Datensatz 0270740
Print as PDF