Problem with subtitles that are sideloaded along with ones defined in the manifest

[OneManStudioDotSe]

I have a problem with the subtitles of videos when playing them through cast.

The videos come with a manifest file that contains entries for subtitles which unfortunately are in a format that is not supported by Chromecast. For that reason, I sideload subtitles which are usable from the Cast framework.

The problem that I am facing is that even though everything seems good and the only available and usable subtitles associated with a MediaInfo entry are the ones that I sideload, when the media actually starts playing through Cast, the entries that are defined at the manifest show up at the dialog for closed captions.

I have tried removing all the available subtitles by passing an empty array at the setActiveMediaTracks() method while being at the ExpandedControllerActivity but the entries are still there at the dialog.

What am I missing?
How can I disable/remove the entries for subtitles that are not supported and only show some of those entries there?
How can I get a hold of the dialog that shows up the available subtitles and handle its content?
is it possible to do any of those things?

I have spent 2 days trying to work my way around this, but with no avail. Any help or tip would be greatly appreciated.