Sort hashtag videos by created_at attribute

[KeshavSharmaWeb]

Hi,

Is it possible or some API is available where I can sort the hashtag videos?

[batmanscode]

@KeshavSharmaWeb you could load the data into a pandas dataframe and filter things that way

[KeshavSharmaWeb]

The data is HUGE. For #cringe, there are about 9lac videos. And we will be having around 1000 hashtags.. if we try to filter like that, we will be in big trouble

[batmanscode]

It would definitely be better to have some sort of filtering before the scrape. For example twint has a min_likes parameter so it only scrapes tweets above that

It doesn't seem like this is possible with this library at the moment. For a large dataframe like that pandas can process things in chunks and alternatively you can try out dask and polars if you want to try out something faster

Personally, pandas has been enough for even the largest dataframes I've worked with which was ~800k rows of bioinformatics data

[davidteather]

+1 you shouldn't have a problem sorting that many