Pathways, an adaptive platform for learning and research

Learner

Manifested
Inputs Perception Processing
Behavior

LEARNER
Advisory Engine SATISFACTION

Clustering
Learning
Opportunity Observed
Patterns
Tests,
Questionnaires,
Implicit profiling

Classification

Pedagogical
Model

Discrepancy Analysis FILL

Website Comprehensibility

Determining an index for

information and instructional value
 Comprehensibility Project
• UASC and MIS collaboration
• Determine whether an automated process
can be used to source and evaluate web
site pages for usage as learning material
• Web presents a challenges for the self-
directed learner
 Method
• Used four professional librarians to source
and evaluate websites
• Focus on STEM topics
• Use golden standard for human judgment
to determine a computational model to
simulate human judgment
 Defining Comprehensibility
• Extraction of instructional/learning value of
the information content in a web site
including the related links
• The degree to which a web page provide
direct access to the substance of the
information of hypertext space without
distractions
 Defining General Public Audience
• Average reading level 8th grade
• Assumed to have internet access
• Internet and Information Literacy is not
assumed
• English reading proficiency
• The population that is served by Librarians
 STEM Topics
• Science, Technology, Engineering, and
Math
• Used 4 taxonomies
– Flandrau Science Center
– National Academies Press
– Library of Congress
– UNESCO
 Website Selection
• 800 STEM sites selected by librarians
• Varied quality levels
• Not all topics are well represented on the
web
• Used many searching strategies to find
content
• Eliminated many non educational websites
 Site Evaluation
• Skim or Read the site content with 10
minutes, click on related links as needed
to understand the site content
• Rate site within 5 minutes
• If prior knowledge of topic or familiar with
the specific website it was eliminated from
evaluation
 Site Evaluation….
• Concept recall to classify the website
• Overall information and instruction rating
• Detailed Rating
– Information Value (text content)
– Information Credibility (credibility)
– Media Instructional Value (visual design clarity)
– Affective Attention (overall appeal)
– Organization (structure, navigation)
– Usability (functionality, interactivity)
 Research Data
• 800 STEM website selected and
evaluated by 4 professional librarians
• 160 web page characteristics for each
website entry point being extracted and
analyzed
• Info space model to define probability of
traversal
 Model Implementation
• Information Value – Heuristic
• Information Credibility – Pattern matching
• Media Instructional Value – Heuristic + Pattern
• Affective Attention – Pattern
• Organization -- Heuristic
• Usability – Heuristic
 Challenges
• What is comprehensibility of web content?
• How do we define the information space?
• How do we define the information and
instructional value of a website in a
manner that may be computed?
• Computational problem of determining
cognitive information processing and
aesthetic reaction to content within the
context of learning objective
 Research Extension
• After data analysis we will create an initial model, apply it
to the content then retest the index
• Gather Learner demographic information
• Connect learners to STEM content to gather feedback
data and compare with Librarians evaluation
• Expand content sampling through TPPL participation
• Cluster learner and content to define research questions
more specifically
• Include task based analysis of scaffolding features