Auto-detection of highlights for clips
After a stream, I’d love to get a selection of moments worth watching and turning into clips, so I don’t have to rewatch the entire VOD manually.
The app should notice when the chat starts reacting significantly more than usual: lots of messages, emotes, or laughter. It would save a segment with a buffer before and after the reaction, so the clip includes the event itself, not just the discussion about it.
After the stream, you open the selection, watch the suggested moments, adjust the start and end points, and save the ones you like. You can delete the bad ones. No auto-publishing everything.
I’d like to be able to adjust the sensitivity, the length of the segments, and account for stream delay. Multiple spikes around a single event should be merged so I don’t end up with a dozen identical clips. In the future, we could add the streamer's voice reaction as an additional hint.
Developer's reply
We can start by finding moments based on spikes in chat activity relative to its usual pace. This is a hint for selection: intense chat activity doesn't always mean something interesting is happening on screen. That’s why we need a post-stream review of candidates with the ability to adjust boundaries and choose what to save.
For the video, you will need to have recording enabled in advance or a buffer of the last few minutes. You can't reconstruct past video from chat history alone. We will need to account for the delay between the event on stream and the viewers' reaction, merge nearby triggers, and limit the storage space used.
Voice reaction can be added in the next stage. Analyzing the game itself via AI should be considered separately, after checking quality and load. In the first version, the goal is simpler: to reduce the time spent reviewing the stream and help find candidates for clips.
No comments yet.