This is why it’s important to have API access available (for a reasonable price, let alone free). The people who want Twitter data are now loading tons of completely unnecessary and heavy multimedia content just to parse it manually for the comparatively light text data they care about. It would literally save Twitter money to give the data away.
There’s a difference between iterating on your code patterns until you get something working and iterating on the solution as requirements evolve. You’re only two months in so you still have loads to learn, but eventually you’ll come to know all the “best practices” so that when you’re presented with a problem, you can immediately know what the ideal solution should look like. The caveat there is that the ideal solution is subject to change as the situation changes.