Same pipeline, only half of it failed
My automated reels publishing pipeline showed an odd pattern. Images and reels with ASCII slugs published normally. Every reel with a Korean slug failed, which covered the zodiac and ilju content.
The way they failed was odd too. Nothing stalled partway through processing. When the media container was created and the platform went to fetch the video at ingest, the status dropped straight to ERROR. It looked like a problem with encoding, duration, or file size. But what decided pass or fail wasn't the content type. It was the characters in the slug.
When your pipeline hands a URL to an outside service, does the path contain raw non-ASCII characters?
The browser had been doing the work for me
On my side the Korean-path URLs looked fine. Pasted into a browser they opened, and the files were on the server. That's because the browser encodes the path before sending the request.
The platform's media fetcher doesn't do that. Given a video_url with raw UTF-8 in it, it didn't try to fetch the file. It rejected it on the spot. ASCII slugs have nothing to encode, so they never triggered the problem, and the bug only showed up on Korean content.
Choosing where to fix it
What would you do: rename the slugs to ASCII, or fix the place that hands the URL over?
I went with the second option. Video, image, and cover URLs all go through a single publish function. At that point I applied urllib.parse.quote to the path part of the URL only.
from urllib.parse import urlsplit, urlunsplit, quote
def encode_media_url(url):
p = urlsplit(url)
return urlunsplit(p._replace(path=quote(p.path, safe="/%")))The important part is safe="/%". Keeping / preserves the path structure. Keeping % stops already-encoded URLs from being encoded a second time. Without that guard, double encoding would break URLs that already worked. I locked this behavior in with a separate test file.
Because the fix lives in one place, video_url, image_url, and cover_url were all covered at once. I checked it with a real publish: a Korean-slug reel's container reached FINISHED and the reel went live. The whole change was 11 lines added to the publish module plus a 26-line test.
Self-check
- Do the media URLs you send to outside APIs carry Korean or other non-ASCII characters in the path, unencoded?
- Have you only been splitting failures by content type, and never by the characters in the slug?
- Is URL encoding done once, at a point every publish path goes through, instead of separately at each call site? And have you confirmed that already-encoded URLs don't get encoded again?
The honest part
I don't know the platform's exact rule for rejecting raw UTF-8 paths. All I confirmed is the result: passed as-is, it fails with an immediate ERROR; with the path encoded, it reaches FINISHED. Because % is in safe, a literal % in a path that isn't part of an escape would pass through unencoded. Query strings were outside the scope of this fix. And I never counted how many Korean-slug reels failed to publish before the bug came to light.