MIT’s OpenCourseWare uses MIT’s Google Search Appliance (GSA) to search its content.  MIT supports customization of GSA results through XSL transformation.  This post describes how we plan to use GSA to search lecture transcripts and return results containing the lecture videos that the search terms appear in.  Since OCW publishes static content, it doesn’t incorporate an integral search engine.  Search is provided through [click to continue…]

{ 0 comments }

How Google Translate Works

by Brandon Muramatsu on August 12, 2010

Google posted a high level overview of how Google Translate works.

Source: Google

{ 0 comments }

Running the Baseline Recognizer

by Peter Wilkins on August 6, 2010

The software that processes lecture audio into a textual transcript is comprised of a series of scripts that marshall input files and parameters to a speech recognition engine.  Interestingly, since the engine is data driven, its code seldom changes; improvements in performance and accuracy are achieved by refining the data it uses to perform its tasks.

There are two steps to produce the transcript.  The first creates an audio file in the correct format for speech recognition.  The second processes that audio file into the transcript.

[click to continue…]

{ 0 comments }

An interesting hack from Yahoo! Openhack India

July 28, 2010

Sound familiar? Automatic, Real-time close captioning/translation for flickr videos. How? We captured the audio stream that comes out to speaker and gave as input to mic. Used Microsoft Speech API and Julius to convert the speech to text. Used a GreaseMonkey script to sync with transcription server(our local box) and video and displayed the transcribed [...]

Read the full article →

Converting .sbv to .trans/continuous text

July 24, 2010

As a step in comparing the output from YouTube’s Autocaptioning, we need to transform their .sbv file into something we can use in our comparison tests (a .trans file). We needed to strip the hours out of the timecode, drop the end time, and bring everything to a single line. Update: It turns out we [...]

Read the full article →

Caption File Formats

July 19, 2010

There’s been some discussion on the Matterhorn list recently about caption file formats, and I thought it might be useful to describe what we’re doing with file formats for SpokenMedia. SpokenMedia uses two file formats, our original .wrd files output from the recognition process and Timed Text Markup Language (TTML). We also need to handle [...]

Read the full article →

SpokenMedia at T4E 2010 Conference

July 14, 2010

Brandon Muramatsu presented on SpokenMedia at the Technology for Education 2010 Conference in Mumbai, India on July 1, 2010. Source: Brandon Muramatsu Download Video (MP4, 230MB) Implementing SpokenMedia for the Indian Institute for Human Settlements View more presentations from Brandon Muramatsu. Cite as: Muramatsu, B., McKinney, A. & Wilkins, P. (2010, July 1). Implementing SpokenMedia [...]

Read the full article →

Towards cross-video search

June 21, 2010

Here’s a workflow diagram I put together to demonstrate how we’re approaching the problem of searching over the transcripts of multiple videos and ultimately returning search results that maintain time-alignment for playback. You’ll notice I included using OCW on lecture slides to help in search and retrieval–this is not an area we’re currently focusing on, [...]

Read the full article →

Making Progress

June 17, 2010

In the last month or two we’ve made some good progress with getting additional parts of the SpokenMedia workflow into a working state. Here’s a workflow diagram showing what we can do with SpokenMedia today. (The bright yellow indicates features working in the last two months, the gray indicates features we’ve had working since December [...]

Read the full article →

Using Lucene/Solr for Transcript Search

June 16, 2010

Overview In any but a trivial implementation, searching lecture transcripts presents challenges not found in other search targets.  Major among them is that each transcript word requires its own metadata (start and stop times).  Solr, a web application that derives its search muscle from Apache Lucene, has a query interface that is both rich and [...]

Read the full article →