Refining the Sample – Description Analysis, Viewing and Programme Exclusion
After exclusion of duplicate programmes, the sample may still require additional refinement. For example, the overall quantity of programmes may remain too large for analysis to be feasible or appropriate with the timeframe of the project. Alternatively, programmes from within the selection may yet prove to be insufficiently relevant to the theme.
Description analysis
One approach that can be taken to reducing the pool of programmes is a description analysis. Using criteria that may be developed from reviewing the literature surrounding your topic, programmes can be included or excluded based upon whether the description adequately illustrates them. Continuing with the example of Henry VIII and war, criteria could be:
However, there are disadvantages to this approach. Carrying out a description analysis before any viewing of the programmes could exclude some relevant content – programme descriptions can differ in how detailed they are about the contents of the programme, with some not having descriptions at all. Analysing programme descriptions is simply a methodology that can be used for short-listing and ring-fencing large datasets, in particular. The more rigorous approach, therefore, is to view a broader range of programmes, but with your exclusion and/or inclusion criteria in mind.
Viewing
There are potential additional inclusion or exclusion criteria to consider when viewing the programmes. For example, how much of the programme actually covers the topic? Drawing comparison, once more, to analysis of print media, the percentage of an article focusing on the topic can be used as a constraining factor. For example, in their analysis of newspaper coverage Zimmerman et al (2019) only included articles if at least 1/5th of the main body text focussed on specific criteria.
We are cautious about applying a percentage factor in relation to broadcast media. For example, a relatively rich but short section of a three-hour breakfast news show might be inappropriately excluded if it did not reach a percentage threshold. We do feel, however, that a minimum number of minutes focusing on a topic may sometimes be a suitable criterion for inclusion. This is also a good point to make note of the timings of any relevant sections of the programmes – both to aid others who may be reviewing your inclusions or exclusions and to refer back to if making clips.
Regardless of the approach taken, researchers ought to be conscious of the potential for bias in both creating and applying inclusion/exclusion criteria. This is particularly true if only one person is responsible for these aspects. Where possible, this risk should be reduced by using assessments made by multiple people.
Transcription generation and detailed analysis of final results
Figure 23: Most TV shows have a transcript available
Transcripts
Having a copy of programme transcripts is both good academic practice (keeping a record of your findings) and provides you with another format from which to analyse programme content. Where transcripts are available (see the above image for where to find them on BoB), they can be copied into a Word document, or depending on the nature of your research, qualitative analysis software such as NVivo.
It is essential to subsequently read through the transcript whilst re-watching the programme, since there will likely be errors, spelling mistakes and/or repeated sentences, especially if the transcript was captured from a live broadcast rather than a pre-recorded show. There may also be unwanted material captured before or after the specific programme, and potentially advertisements during transmission. Errors and unnecessary inclusions can then be corrected in the Word document (however, it is not possible to correct the original BoB transcript due, amongst other factors, to licensing agreements).
Programme analysis
Transcripts are a powerful resource for both selecting and analysing programmes. However, for AV materials viewing and/or listening remain crucial. This is especially true if you wish to exploit some of the depth that visual representation can offer (see introduction for further examples of research questions).