Transforming and publishing official Parliamentary transcripts is one of the key activities of parliamentary monitoring organisations (PMOs) — in our case, that means running our website TheyWorkForYou, but there are many organisations around the world doing the same for their own parliaments.
Building on top of transcripts means that PMOs can focus their time on where they can add value to those transcripts: either applying their digital skills to make them more accessible, or merging them with other datasets and sources in ways the official Parliament sites cannot.
One of the interesting things about Parliamentary transcripts is that they don’t exactly reflect what happened. They can be an official record of what was supposed to have been said rather than what was said: a constructed version of Parliament that is close to, but not exactly, reality.
This can be very important when democratic needs are not for verbatim transcripts. MPs can ask to make corrections on factual content if they misspoke. The Record can add useful shorthand, referring to standing orders, or additional information that was not said orally. For parliaments with multiple official languages (in the UK’s case the Senedd/Welsh Parliament), transcripts make parliamentary activity accessible in all official languages.
But the difference can also be political in the “first draft of history” sense. The transcript can retrospectively apply rule-following in a way that can remove political speech on the edge of those rules. We’ve noted before there are times where the transcript does not reflect reality in procedurally significant ways. Historically famous events can be at the edge of the transcript, rather than visible in it.
So there are pros and cons to our understanding of parliamentary activity being dependent on transcripts alone. But the existence of accessible video recordings of Parliament makes it easier than ever to see the difference between what officially happened and what actually happened.
Combining transcript and video
In 2008 we did some work with the to explore how to make parliamentary video searchable. At the time the best approach we had was a crowdsourcing approach to reconcile timestamps and speeches — around 400 volunteers aligned 160k speeches. Ultimately this process had big technical overhead in video storage, and a lot of manual work was required to keep the two feeds together, and we stopped using video in this way.
New approaches make this possible at scale with much less manual effort. As part of our TICTeC Community of Practice around Parliamentary Monitoring we ran a session on video and transcripts, hearing from OpenParliament.tv about their approach (video of the presentations).
OpenParliament.tv currently covers the German Bundestag, but is interested in expanding the approach to more countries. The platform is a combined video/transcript search platform, where individual speeches can be searched for, and jump the video to that point in the record.
There’s a lot of moving parts that make this work, but the core work is in how the video and the transcript are aligned. The transcript is converted to computer-generated audio, and then the generated audio is matched against the real audio. The matching uses an adapted version of the aeneas framework for ‘forced alignment’ of text and speech.
When you think about it, this approach makes a lot of sense: speech to text is often specifically bad at generating the punctuation of written language, while one of the key things in syncing transcripts to video is finding the start and end of blocks. This can still run into difficulties when what is being said is just not present in the transcript, but generally it can flow around problems and match items on either side. From an international perspective, this is also interesting in that it’s an approach that works better across different languages than speech-to-text approaches.
In other technical details, Open Parliament TV does not host the video themselves. Offline they need to process it to extract and match audio, but online their player links to the videos as hosted by the Parliament. In the long term, they want a workflow that sends videos to an internet archive as a backup. This is another useful purpose of democratic transparency project: – to act as civic redundancy against backsliding access to democratic materials.
Switching between modes
We had a bit of a discussion in the group about transcripts, and about where AI approaches might make it possible to generate them. This might be useful in settings where there is only video output, to make it easier to search and parse. This could also be a useful bridge in cases where there is a significant delay before transcripts are released.
But official transcripts are an art beyond just writing down what’s happened, imposing a consistency on the parliamentary record that is very useful as a building block connecting it to other data. For instance, Open Parliament TV does additional detection of named entities based on the transcript, meaning that specific mentions can then be seen in the video.
The future of parliamentary monitoring might be switching between these modes: making the ground truth of what happened visible through video, augmenting this with the formal transcript, and bridging from that to other sources of information. In short, pulling on what different mediums do best to make democratic processes stronger and more transparent.
Header image: Photo by Diego González on Unsplash
