FYI Literacy

Currently, everything I read is grant related. Well, almost everything… I just bought a Barnes & Noble Nook (e-reader) and I've begun re-reading The Hitchhiker's Guide to the Galaxy which is really taking me back to the 1980's. Some things never go out of style, such as this sage advice for intergalactic hitchhikers: don't panic. I'd have to say it has served me well, even in my current work assignment as Title V project director. Not to panic; that would be a "best practice." I also used to read quite a bit of philosophy in my undergraduate days. Epictetus (AD 55-135) has always been one of favorites, for his practicality, his notions on power and control, and also acceptance. This quote from him is particularly compelling: "It's not what happens to you, but how you react to it that matters." In these terms, our colleagues, Kelly Lambert and Robin Cotter, have wrung additional value from their ongoing project developing information literacy modules; specifically in the manner in which they have responded to the challenges of assessing student learning outcomes.

Not to panic; that would be a "best practice."

As part of the interdisciplinary "Biology Bootcamp" series, the information literacy component aims to assist students in developing study skills that will enable them to successfully complete their course. In fact many of the Bioscience courses have substantive information literacy elements: BIO108 (research and presentation), BIO181 (report on infectious diseases), BIO182 (paper on evolution, including website evaluation; research on water quality and citing research), and BIO205 (researching and preparing a brochure on a genetic disorder). As a result, the information literacy tutorials have broad utility among most sections of bioscience classes. Furthermore, they can be adapted for other science and non-science courses. If you know of or are part of another academic department that would like to learn how this Title V-funded resource can be adapted to more specifically meet the needs of your students, please let me know. 

In the first phase of this project, undertaken and completed in fall 2010 with additional and significant contributions from Breanna Prinzhorn (bioscience faculty), content was created for the first three modules, including videos, transcripts and quizzes. Pre- and post-assessments were also created, tested and revised. Options for entering pre- and post-tests into Lesson Builder and Blackboard were also explored, but more about that shortly. Below is an embedded video from the tutorial (one of my favorites) and below that is a web link to the project as it appeared at the end of phase one. On behalf of her team, Kelly presented on phase one of the project to the full Title V Steering Council in November 2010 and received high praise.

http://www.pc.maricopa.edu/ctlt/titleV/Science/InformationLiteracy/index.html

Revised modules (phase two) can be viewed at Robin's BIO205 website, "MicroMania" (select from modules 1-8 in the column to the right).

Now about that assessment data… it turns out that one of the elements of this project -- a minor detail, really -- turned out to be quite a chore and continues to pose a challenge to implementation. One of the project tasks was to explore options for testing students and then harvesting that data to not only assess learning outcomes for the instructors but to also feed that information back to the students themselves. Due to interface issues between Soft Chalk (Lesson Builder) and the now-older version of Blackboard (Phoenix College is not using the latest version), pre- and post-test data cannot be easily uploaded, disambiguated, or otherwise used to take advantage of Blackboard features such as active release. Other options may allow more flexibility but then student authentication, controls on cheating, "time on task" authentication, and autoscoring must be sacrificed. To date, this component of the project has been constructed in Google docs, Survey Monkey, Lesson Builder, and Blackboard. Each solution provides an important piece but none of them offer a perfect solution. Kelly and Robin have navigated this challenge with aplomb. Their response to this challenge, and ongoing work with Jan Binder, has not only revealed a limitation in the assessment of resources such as these but is also moving us closer to potential solutions that will allow us to not only say that our work has made a positive impact on student learning outcomes; it will allow us to demonstrate it. Until then, don't panic.

By the way, for those of you who are looking for a little reading material, the pre- and post- test surveys can be viewed via the following URLs:

• Pre https://www.surveymonkey.com/s/N56TDHZ 
• Post https://www.surveymonkey.com/s/F8KJQ6G

Goals, Objectives, and Tasks! Oh my!

Grant evaluation is nothing like a scary walk in the woods but there is an awful lot of repetition of and return to the overarching goals of the project, it's underlying activity objectives, and the specific tasks that demonstrate incremental progress. It amounts to a cumbersome mantra. Oversight and coordination of the evaluation process rests with the Title V Steering Committee. As designated in the grant, they are: the vice president of academic affairs (Casandra Kakar), the vice president of student services (Yira Brimage), the project director (Seth Goodman), the institutional researcher (Jan Binder), the coordinator of the success center (Diana Mitchell), a faculty member (Mark Rosati), the college grants liaison (Sharon Halford), and the external evaluator. The external evaluator for the grant is William Baker and over the past two years, he and I have become increasingly relaxed during site visits as evidenced by our attire. In the first year, for both of us, suit and tie all the way, the entire day. At this point, Dockers and a dress shirt for me. For William, no tie and the suit jacket gets draped across a chair on arrival.

Yesterday was Mr. Baker's fifth visit to Phoenix College, to check on our progress, to meet with the Steering Committee, and to talk about goals, objectives and tasks. There were no flying monkeys. The day was roughly divided up into three areas of review: the STEM Scholars program, the status and overall progress of implementation activities, and the assessment strategies for demonstrating the impact of grant activities on student outcomes.

During the morning, Mr. Baker met with Laura Torres, the interim Math and Science Specialist, to gain a deeper understanding of the week-to-week activities she is involved in with supporting our cohort of STEM students. She pulled out a 4-inch binder and walked him through the activities of the cohort, her strategies for engaging them in campus life activities, and supporting them in developing their assets for college success and career exploration. We were fortunate enough to have five STEM Scholars visit with us and share their personal experiences at Phoenix College and in the program. They all had profoundly positive remarks to share about Laura's proactive involvement and support and the benefits of the program and the resources it provides to them. The session transformed into a brainstorming session for prospective improvements going forward into the 2011-12 school year. The students take the cohort program very seriously and demonstrated this with their eagerness and willingness to improve it for the next wave of students to join their ranks. "Wow… that's fantastic… I'm really impressed…" Mr. Baker started to sound like a broken record. Kudos certainly go to Laura for this accomplishment. She came in at the start of the semester with no time to prepare. She hit the ground running and won the trust and confidence of our student cohort one member at a time.

Ongoing progress is being made in most, if not all, objectives. Underspending of funds and implementation data for mini-grant projects are areas where improvement and accelerated progress are needed. We explored the status of reporting for mini-grantees, payment procedures for contracts, and other institutional factors and policies with an aim to improve implementation by the end of Year Three, in September. The site visit concluded with a formal meeting of the 8-member Steering Committee, a subset of the more commonly known Title V Steering Council, which also includes a panel of 11 faculty advisors and a select group of 9 resources members. Considerable discussion centered on the reporting and framing of retention and persistence data and correlating differing levels of cohort data. Some additional strategies may be explored to demonstrate progress and validate improvement in the coming year. Additionally, there will be a push to acquire more complete information on the deliverables from faculty mini-grant projects and the associated student learning outcomes.

The "dashboard" that was very helpful in facilitating conversations about implementation status, progress, and assessment yet received mixed reviews and will likely be revised.  Mark Rosati has a particular appreciation for bar graphs over pie charts.  Color key (generally): green = completed, yellow = on track, and red = lagging.

Near the conclusion of the Steering Committee meeting, Mr. Baker excused himself to meet his taxi and make his way to the airport. Apparently the cab driver couldn't "make it" so I packed Mr. Baker into my CR-V to rush him to his plane. He had many compliments for the day's visit and the staff and faculty involved in the grant. We shifted into some light conversation. He commented on how his golf game is declining… and then we somehow ended up talking about the grant's goals, and objectives, and tasks again.

Mr. Baker will return for another site visit in early fall 2011.