The prevalence of the six-seconds rule may be rooted in the belief that fast subtitle speeds will not allow viewers to follow both the subtitles and the on-screen action . However, how much time do viewers actually spend reading subtitles and watching the images This can be assessed using the concepts of absolute reading time and proportional reading time . Absolute reading time is measured in seconds and it is the actual time spent on reading the subtitle. For instance, a viewer can spend 4 seconds reading a subtitle displayed for 6 seconds, which leaves them 2 seconds to follow the on-screen action in the film. Proportional reading time is measured in percentages and is the proportion of the total subtitle display time during which the viewer is actually gazing at the subtitle. Thus, if a reader looks at the 6-second-subtitle for 4 seconds, their proportional reading time is 66%. Longer subtitle display times have been found to increase the absolute reading time but decrease the proportional reading time [15, 16]. On the one hand, this finding may suggest that longer subtitle display times can benefit viewers by giving them more time to follow the on-screen action. On the other hand, however, it is plausible that when faced with fast subtitles, viewers simply read them more efficiently and, ultimately, do not need longer display times.
The reading process of subtitles is dependent on whether viewers can understand the language of the film soundtrack. If they do, it is plausible to assume they would spend less time reading the subtitles, as they would largely draw on the auditory information in their processing of the film dialogue. However, should viewers be unfamiliar with the language of the film soundtrack, we may expect that they would rely more on the subtitles and their subtitle reading time would be longer.
Given that European TV channels and cinemas are largely dominated by English-language productions , a large volume of subtitles on the audiovisual translation market are translations from the English language . English is also the best known and the most studied foreign language in the EU . Taken together, this means that many viewers are not only able to understand what is being said in the film without any translation but can also compare the original English dialogue with subtitles. Yet, although in theory such viewers do not need subtitles to follow the dialogue, it has been found that they read subtitles anyway [47, 52]. This phenomenon has been attributed to the dominance of the visual modality , the dynamic nature of moving subtitles as they quickly appear and disappear on screen [47, 53] and to the attractiveness and saliency of text even if it is presented in an unfamiliar language . Considering the ubiquity of subtitled content in the world today, it is important to determine the exact role of the language of the film soundtrack on the processing of subtitled videos.
When it comes to the differences between the videos in a language that is familiar (English in Exp. 2) and unfamiliar (Hungarian in Exp. 1) to viewers, we hypothesized that because people support their viewing with auditory information from the soundtrack, the preference for faster speeds and unreduced text may be more discernible when they understand the language of the film dialogue, whereas it may be less pronounced in the case of a language that viewers have no knowledge of. Furthermore, the analysis between different groups of subjects (Spanish, Polish and English) enabled us to consider the impact of experience with subtitling on the processing of subtitled videos. We expected that people who are familiar with subtitling may have developed certain strategies allowing them to process subtitles more efficiently, possibly evidenced by higher comprehension and lower cognitive load.
A powerful combination of different research methods: eye tracking, questionnaires and semi-structured interviews, has enabled us to isolate the impact of speeds on the processing of subtitled videos modulated by different linguistic backgrounds of viewers. To the best of our knowledge, no work to date has investigated subtitle speeds using such mixed methods approach. Our approach provides a unique research opportunity to determine whether modern viewers are able to keep up with fast subtitles and to measure their viewing experience in relation to different speeds, the language of the soundtrack and their familiarity with subtitling. Investigating these issues is particularly useful in the context of the multiplicity of subtitled content and current subtitling practices.
Despite our expectations prior to the study and the linguistic background of the participants, when asked about the preferred type of audiovisual translation, the vast majority stated they prefer subtitling. This, on the one hand, may reflect changes in audiovisual translation landscape, and on the other may be attributed to the fact that the participants were living in the UK at the time the study was carried out. Finally, the preference for a given type of translation is not synonymous with its prevalence in a country; this is to say that although some participants may prefer subtitles now, they still grew up in a non-subtitling country.
The dependent variables were: comprehension score, three indicators of self-reported cognitive load (difficulty, effort, frustration), enjoyment, scene recognition, subtitle recognition, and five eye tracking measures. We also collected categorical data on reading experience. Similarly to subtitles, all the questions were presented to the participants in their native languages.
To assess how participants coped with different subtitle speeds, we asked them if they had enough time to read the subtitles, if they re-read subtitles, if they missed words from the subtitles, if they had enough time to read the subtitles and follow the on-screen action and, finally, what subtitles they prefer: verbatim or condensed.
Expecting that fast subtitles would be more difficult, effortful and frustrating for viewers to process, we asked the participants to assess the difficulty of the subtitles in the clips as well as the effort they had to expend in watching them, and the level of frustration they experienced. We found the main effect of speed on difficulty and effort but not on frustration (see Table 5). Participants generally declared the lowest cognitive load in the case of slow subtitles (12 cps). There were no interactions.
Based on the assumption that if subtitles are too fast, viewers may not be able to read them, we asked the participants to recognize phrases from subtitles. We predicted that if people did not manage to read a subtitle in its entirety, their ability to recognize the subtitle wording would be hampered. However, the impact of speed on subtitle recognition did not reach statistical significance, F(2,140) = 2.529, p = .083, = .035 (see Table 9). There were no interactions.
Subtitle speed had an effect on all eye tracking measures (Table 10). There were no interactions. Slower subtitles induced more fixations and higher mean fixation duration than faster subtitles. The absolute reading time was longest in the 12 cps condition, whereas the proportional reading time was highest in the 20 cps condition. Fig 1A shows that an increase in subtitle speeds resulted in an increase in the percentage of time spent in the subtitle area, relative to subtitle duration. Subtitles in the slowest condition (12 cps) triggered the largest number of revisits, which may mean that participants read the subtitle, looked at the scene and gazed back at the subtitle area, only to find the same subtitle there. We discovered a trend, depicted in Fig 1B, that the longer the subtitle duration, the more revisits to the subtitle area. When watching slow subtitles, viewers re-read two out of three subtitles, but when watching fast subtitles, they re-read about one in five.
There was a main effect of language on all dependent variables apart from revisits (Table 11). Post-hoc Bonferroni tests showed that Polish participants spent the least amount of time in the subtitles and differed significantly from Spanish participants in all eye tracking measures (fixation count, p = .029, 95% CI [-1.26, -.05]; mean fixation duration, p = .001, 95% CI [-56.23, -11.34]; absolute reading time, p < .000, 95% CI [-491.85, -156.68] and proportional reading time, p < .000, 95% CI [-.20, -.06]) and from English participants in the case of absolute reading time (p = .027, 95% CI [-351.12, -15.96]). Overall, Spanish participants dwelled longest in the subtitle area and their fixation duration was the longest, indicating highest effort among all the groups.
Reading experience questions showed that participants in general could cope well with all the three speeds. Most participants declared that subtitles were displayed for the right amount of time (Fig 2) and that they had sufficient time to read them as well as to follow the on-screen action (Fig 3).
Contrary to our expectations, subtitle speed did not affect comprehension, scene recognition, or enjoyment. As demonstrated by reading experience questions, participants were equally satisfied with the speed of subtitles in the slow, medium and fast conditions. In all speeds, participants declared to have had sufficient time to read the subtitles and to follow the on-screen action.
Overall, the highest cognitive load was reported by English participants, who are generally not accustomed to subtitling. In contrast, Polish participants declared lowest frustration among all the three groups. The self-report results were confirmed by eye tracking as they also spent the least amount of time in the subtitles and had lowest mean fixation duration, indicating lowest cognitive load. Despite the results from the self-reported cognitive load and enjoyment, eye tracking data showed that Spanish participants may have experienced higher cognitive load as they had highest mean fixation duration and spent the highest time reading the subtitles, as shown by both the absolute and proportional reading time. Their comprehension results were also the lowest. 59ce067264