Join us and get access to thousands of tutorials and a community of expert Pythonistas.
This lesson is for members only. Join us and get access to thousands of tutorials and a community of expert Pythonistas.
Choosing Chunk Size
00:00 Till now, you’ve used a specific number, that is, 100 kilobytes, as the chunk size. You might be wondering why we chose that number and how it affects your script.
00:09
Choosing the correct chunk size is quite important, and that’s what we’re going to look at next. When you download files with Python’s requests library, you control how much data you read at a time by setting the chunk size.
00:22 For small text or JSON files, you can keep the chunks tiny, around 4 to 8 kilobytes, because the files are lightweight and don’t need much memory. For medium-sized files, like images, documents, or short clips, you’ll get better performance if you increase the chunk size to somewhere between 32 and 128 kilobytes. And when you’re working with large binary files, such as videos, backups, or datasets, you should go even bigger.
00:48 Use chunks between 256 kilobytes and 1 megabyte. In this example, you’re streaming a video and reading it in 512 kilobyte chunks, which is a more efficient choice for large files and helps your download run faster and smoother.
01:03
Now you know how to download files using urllib, requests, and how to stream your file. Next, let’s see how to make the correct choice in choosing between these methods of downloading files.
Become a Member to join the conversation.
