Scraping TikTok

TikTok's Android app talks to a private HTTP+JSON API that is faster than the web endpoints and returns considerably more. This is a technical guide to reaching it: how devices are registered, how requests are signed, how the regional hosts are partitioned, and how the TLS handshake is fingerprinted. A system built on it collected 3.23 billion creator profiles, 5.94 billion videos and 2.8 billion comments in three weeks.

Free dataset. I uploaded 4.5 billion of those videos to Hugging Face: captions, view, like, comment and save counts, the sound, the country and the posting time. huggingface.co/datasets/kuben-developer/tiktok-videos-4b

What you can pull. Creator profiles, every video a creator has posted, followers and following lists, and TikTok's own similar-creator graph. Full video detail with the complete statistics block. Comments and comment replies, each with the commenter's account. Sounds, the videos using them, and the trending sounds chart. Hashtags and their videos, newest or most popular. Keyword search across videos, creators and sounds. Trending shelves and camera effects. 24 endpoints in all , each with a measured success rate.