Find the missing RNAs reported by GeneCArds #210
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…in an earlier groupby I think this is working as expected now - the lists of genes targeted by an miRNA contained duplicates (I don't know why) which led to conflicts on insertion to the databse. This now produces FAR less data, which is somewhat concerning
…raise an exception when the providing database is missing for an unmapped region
|
' |
| @@ -76,6 +78,9 @@ def build(cls, index, raw): | |||
| "databases": clean_databases(raw["databases"]), | |||
| } | |||
| if not metadata["providing_databases"]: | |||
| if not raw["was_mapped"]: | |||
| print(raw) | |||
There was a problem hiding this comment.
Probably better to change the exception to f".. {raw}" to show what data has the
issue.
|
Poking around a bit and I can't see any other obvious sources of problems. So probably a combination of what you found and some other issues. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Between releases 23 and 24, a large number of RNAs went missing from our genome coordinate FTP exports. GeneCards reported this for our human coordinates, and gave this table that summarises what went missing:
This amounts to a little over 400k sequences that have gone missing.
One possible reason is badly handled export of mapped vs recieved coordinates. When an expert db gives us coordinates, we import them and they get associated with the database when we export them, and have
source=expert-databasein the gff. Mapped sequences should havesource=alignmentin the gff, and inspecting the release 24 gff file - there are 0 aligned sequences, which is clearly not right.I've modified how we handle the mapped sequences in the gff writer so that we only throw an exception if :
This is able to produce gff files that contain ~273k aligned sequences that were previously missing. I think this is part of the missing data, but we still need to find ~100k more, so this is not completely finished yet.