Worried about how to evaluate software before you buy? Consider these seven tips before finalizing that PO.
By David Siminoff
If you surf www.SnakeOilSalesmen.com you will find the pantheon of educational technology products that didn’t work. Lots of promises. Lots of noise. No tangibly good results for our kids. Waste.
It’s a story replayed too many times in our districts, and the world of technology is getting more complex, more ripe for abuse.
When you buy tech, you wear two hats—you play both offense and defense. You are trying to let your students take advantage of truly new things (e.g., iPads), not rehashes of the past (“new” paper textbooks). You take risks: Will the technology work? Will it actually teach…better? Will it deliver the value it promises in helping our children find dots of light in the darkness of educational space?
You mitigate risk as well: If you play it safe, you…don’t get fired. (See the guy who managed the LAUSD iPad debacle for details.) You order paper texts—they don’t need batteries. You…do the same thing your predecessors have done for 300 years. The kids get their education, more or less. And they move on. Defense.
The “right” answer probably circles a balance of both offense and defense. I present to you a series of what I believe should be key questions you ask of your vendors in defining that space:
1. Can you spot “lazy” tech?
Is the “brand new” technology just an old paper textbook whose pages have been scanned, copied. and pasted onto a vanilla web page? Brilliant. Cutting-edge technology. Bazinga.
Is the “new” product essentially the same thing the company offered three years ago, but the cover/title page has been changed to include the words And now with Common Core!? It always had the Common Core. Somebody took a day or two to numerically tag the Common Core grids. Is this a reason to shell out money for a “new” product?
When companies offer “new” products that leave you scratching your head, then it’s likely you are buying an empty basket.
2. Can you believe it's not butter? (Or, is the data real or not?)
Can you trust that the product actually works? Is there quality data, which backs up the supposed findings? (“Quality” means a smart 15-year-old can understand the data.) The sample size was big. There are testimonials from people who actually implemented the product. The product actually worked—i.e., it wasn’t just a “paid study.” And the post-tech-application result was demonstrably better than the control group’s. If the product is really good, it’ll let you track to the individual student, the classroom, the area of study, and time spent on the platform.
3. Is this a transaction or a relationship?
Ever feel like you’re being pushed out of the room the minute you shake hands with your vendor? Big companies are publicly traded entities run for shareholder profits. They have sales quotas to hit and pressure to sell, sell, sell. Small companies have pressure as well—but their “greed” is often more of the long-term flavor (healthy) than the short-term flavor (think: your last stockbroker).
Don’t be misled by large companies who claim to have 10,000 service center employees, but when you reach out, nobody returns your calls. In many small companies, you get the founder or top officer who cares about a long-term relationship actually working. Their company’s survival usually depends on it. They’ll do right by you (i.e., do what they say they will do)—or they will die fast.
4. Should you test the system?
Yes. Here’s an easy one: Send an email to the company’s Contact Us button with a clear, reasonably simple question and see what happens. Just because it has a big PR firm/marketing agency that claims to have great customer service doesn’t mean that it actually does. And to be very clever, if you work for a gorilla like LAUSD, don't send the email from that vaunted account. Send it from your personal account and see if you get a quality reply any time soon.
5. Is there fair value in the product?
One of the reasons that I started Shmoop was that I found myself responding angrily to the textbooks my children were bringing home. To wit, the history book was basically the same one I had in high school 30-plus years earlier—little if anything had been added to give context and relevance to the vast changes in society today versus the 1980s. Yet the textbook cost $220. It was still unfathomable on many levels, not Internet-friendly, and used the same $5 words that made the teacher feel smart and the student feel stupid. If the new technology doesn’t feel like a leap forward over whatever was in the past (think: iPad relative to a laptop), it’s usually a pass.
6. Does the vendor truly love its own product?
Love matters. It’ll keep us together, it hurts, and it’s a battlefield. Founders know all of this about their products—they love them. Sometimes the product loves them back, and other people fall in love with the same simple clarity or voice or vision that the product imbues to its category.
Small things matter—is the product actually updated regularly, or does it just say it is? Digital is designed for regular updates; paper not so much. Is the product easy to implement without requiring that the teacher have a Ph.D. in computer science?
Does it serve pages well on an iPad, an old Dell laptop with an Explorer browser, on a Samsung Mega, on a new Mac?
7. The acid test: Do students actually use the product? Do they, perhaps, love it?
If they do, it “will love them back” in nonobvious ways. Take an ungodly boring task like studying for the SATs, or reading Kants, or diagramming orbitals. If the new tech makes that process painless, maybe even fun-ish, then the student, who fights to spend the least amount of time possible on a given task, will spend much more time. And learn more. Perhaps the student will “feel” instead of just “know” the topic.
David Siminoff is the founder and chief creative officer of Shmoop University, www.shmoop.com.
Image: Olaru Radian-alexandru/Hemera/Thinkstock