How to handle large number of .ldb files

[ErickKramer]

Detailed Description

Hi, I have been searching for how to handle/clean old .ldb files stored in the db/ directory but could not find any information about it.

Running aptly db cleanup does not seem to affect this.

I tried simply removing the oldest .ldb file I had and that "broke" the database until I ran aptly db recover.

Are there any best practices or recommendations on how to limit them?

Context

LevelDB files

Your Environment

Debian Bookworm

[neolynx]

How many lcd files are you seeing ?

I don't think there are 'old" lbd files, as the db is incremental. A aptly db cleanup should compact the goleveldb, and all the files are internal to goleveldb.

what aptly version are you runnning ?

[ErickKramer]

How many lcd files are you seeing ?

I don't think there are 'old" lbd files, as the db is incremental. A aptly db cleanup should compact the goleveldb, and all the files are internal to goleveldb.

what aptly version are you runnning ?

Thank you for the reply. I just checked and running aptly db cleanup does indeed decreased the number .ldb files, but just slightly.

I tested using a playground aptly setup I am testing using aptly version: 1.6.0~beta1+20240924144106.96e60ae5 and the files went from 579 to 509.

I would expect the same behavior on the stable aptly setup we have using aptly version 1.5.0. We run aptly db cleanup once a month in there.

So from what you mentioned in regards to the db being incremental, the more packages are tracked by aptly, the larger the number of .ldb files there will be. Is that correct?

[neolynx]

thanks for the explanations.

yes, this is the way leveldb is designed, if there are no performance/inconsistency problems, there should not be any problem with many lbd files.

aptly usess a newer version of goleveldb in the ci builds, I assume it optimized the db compaction and therefore is able to reduce the number of ldb file further.

[ErickKramer]

Thank you for explaining how it works.