Issue with loading updated records from pipedrive source

[sauravganguli]

Source name
Source pipedrive, with the deals endpoint

Describe the bug
Some of the records in the deals table are not getting updated. For example, this deal in the screenshot attached was updated once on 20th February and once on 27th. But these changes were not picked up by during the data loads and in our table we see the last update for this deal was on 6th February, refer to screenshot attached.

As a temporary fix we did a full reload of the table after which these updates were loaded. But we still don't know what causes the issue and therefore we might face the same issue again in a few days.

It's not like updates are not reflected at all, but only partially loaded. So some of the deals changed might be correctly loaded to the tables but some of them not.

1. name of the dlt source causing the problem: pipedrive
2. name if the dlt resource causing the problem: deals endpoint incremental load
3. which destination you are loading into: BigQuery
4. any stack traces and logs you can share: Attached screenshots to understand the issue
   
   

To Reproduce
Make changes to some deals in pipedrive and run the pipedrive pipeline to extract the updated records

Expected behavior
When a change happens in pipedrive for any one of the endpoints, the next run of the pipeline should be able to correctly update all rows in the extracted tables

Stack traces and other evidence
Attached the code files for the pipeline and our DAG to invoke the pipeline
pipedrive pipeline.zip

Running environment

• OS: MacOs
• dlt Version 0.4.2
• Orchestrator: Google Composer