'SwigPyObject' object has no attribute 'thisown' when trying to apply multiple redactions

[willi3by]

Please provide all mandatory information!

Describe the bug (mandatory)

I am trying to loop through a set of patterns to find sensitive text and redact on a pdf. However, after the first time I use apply_redactions(), I can no longer use the page and get the following error:

Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3552, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-6-705099d09e90>", line 1, in <cell line: 1>
    redact_page(path_to_pdf, 0)
  File "<ipython-input-3-ec7073088d3d>", line 55, in redact_page
    sensitive = get_sensitive_data(page.get_text("text").split('\n'), sd)
  File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/fitz/utils.py", line 764, in get_text
    tp = page.get_textpage(clip=clip, flags=flags)
  File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/fitz/fitz.py", line 5584, in get_textpage
    textpage = self._get_textpage(clip, flags=flags, matrix=matrix)
  File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/fitz/fitz.py", line 5571, in _get_textpage
    val.thisown = True
AttributeError: 'SwigPyObject' object has no attribute 'thisown'

To Reproduce (mandatory)

While I cannot share a sample file because it has PHI, below is the code I am using:

This is the function to find the sensitive data

def get_sensitive_data(lines, sensitive_regex):
    for line in lines:
        # matching the regex to each line
        if re.search(sensitive_regex, line, re.IGNORECASE):
            search = re.search(sensitive_regex, line, re.IGNORECASE)
            yield search.group(1)

and this is the code to apply the redactions:

def redact_page(path_to_pdf, page_number):

    NAME_REG = r"(.*,\s*.*\s.*)"
    SPECIMEN_ID_REG = r"(Specimen ID:.*)"
    CONTROL_ID_REG = r"(Control ID:.*)"
    ACCT_ID_REG = r"(Acct #:.*)"
    PHONE_ID_REG = r"(Phone:.*)"
    AGE_ID_REG = r"(Age.*)"
    PATIENT_ID_REG = r"(Patient ID:.*)"
    DATE_COLLECTED_REG = r"(Date collected:.*)"
    DATE_RECEIVED_REG = r"(Date received:.*)"
    DATE_ENTERED_REG = r"(Date entered:.*)"
    DATE_REPORTED_REG = r"(Date reported:.*)"
    ORDERING_ID_REG = r"(Ordering:.*)"
    NPI_ID_REG = r"(NPI:.*)"
    DOB_REG = r"(DOB:.*)"
    RESULT_NUMBER_REG = r"(CFT.*)"
    CLIENT_SPEC_ID_REG = r"(CLR.*)"

    sensitive_data_types = [
        NAME_REG,
        SPECIMEN_ID_REG,
        CONTROL_ID_REG,
        ACCT_ID_REG,
        PHONE_ID_REG,
        AGE_ID_REG,
        PATIENT_ID_REG,
        DATE_COLLECTED_REG,
        DATE_RECEIVED_REG,
        DATE_ENTERED_REG,
        DATE_REPORTED_REG,
        ORDERING_ID_REG,
        NPI_ID_REG,
        DOB_REG,
        RESULT_NUMBER_REG,
        CLIENT_SPEC_ID_REG
    ]

    doc = fitz.open(path_to_pdf)
    page = doc[page_number]
    for sd in sensitive_data_types:
        sensitive = get_sensitive_data(page.get_text("text").split('\n'), sd)
        for data in sensitive:
            areas = page.search_for(data)
            [page.add_redact_annot(area, fill=(0, 0, 0)) for area in areas]
        page.apply_redactions()
    doc.save('redacted.pdf')

    return print("Success")

Interestingly, after the first redaction is applied, if I try to extract another page from the doc, page = doc[0], I get the same error. The only way I can perform another action on the document is to restart the python kernel.

Your configuration (mandatory)

• M1 Mac, MacOS 11.6.8
• Python: 3.9.10
• PyMuPDF version: 1.20.2 from wheel

Let me know if you need any other info.

Thanks!

[JorjMcKie]

Always a problem if there are no data for reproducing an error!

What I did notice looking at your text:
You are applying redactions for the same page several times - which at least is unnecessary and a performance hog.
Also be aware that this function changes the page such, that a new TextPage object of it must be created (which is the basis of all text search and extraction functions) before another function is executed that needs a textpage.
Doing it your way may lead to changing the iterator while it is executing - the exception message seems to indicate exactly that.
So try taking page.apply_redactions() out of any loop and execute it exactly once per page.

[JorjMcKie]

For example, this snippet basically does the same thing: make multiple redactions on all pages and apply them. And it does work:

import fitz
doc=fitz.open("PyMuPDF.pdf")
for page in doc:
    rl=page.search_for("link")
    for r in rl:
        page.add_redact_annot(r)
    if rl!=[]:
        page.apply_redactions(images=fitz.PDF_REDACT_IMAGE_NONE)

'Redact' annotation on page 2 of PyMuPDF.pdf
True
'Redact' annotation on page 4 of PyMuPDF.pdf
True
'Redact' annotation on page 5 of PyMuPDF.pdf
'Redact' annotation on page 5 of PyMuPDF.pdf
'Redact' annotation on page 5 of PyMuPDF.pdf
'Redact' annotation on page 5 of PyMuPDF.pdf
'Redact' annotation on page 5 of PyMuPDF.pdf
True
'Redact' annotation on page 6 of PyMuPDF.pdf
'Redact' annotation on page 6 of PyMuPDF.pdf
True
... and so on ...

[willi3by]

@JorjMcKie Thanks for the response! I agree, if you have any suggestions on generating a sample to send, I am happy to do so. As I said, all the files I have contain patient information, which is why I am trying to do the redaction.

After your comment, I changed the main loop in the code to the following:

def redact_page(path_to_pdf, page_number):

    doc = fitz.open(path_to_pdf)
    page = doc[page_number]
    all_areas = []
    for sd in sensitive_data_types:
        sensitive = get_sensitive_data(page.get_text("text").split('\n'), sd)
        for data in sensitive:
            areas = page.search_for(data)
            all_areas.append(areas)
    [page.add_redact_annot(area, fill=(0, 0, 0)) for area in all_areas]
    page.apply_redactions()
    doc.save('redacted.pdf')

    return print("Success")

I am only working on the first page right now, the loop is to loop over multiple regex patterns for search conditions, so each iteration generates the areas that need to be redacted and they are appended to all_areas. The page.add_redact_annot statement is successful but when I try to apply redactions I get a similar error code:

Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3552, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-14-a297f10aa9d5>", line 1, in <cell line: 1>
    page.apply_redactions()
  File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/fitz/utils.py", line 3993, in apply_redactions
    for annot in page.annots(types=(PDF_ANNOT_REDACT,)):  # loop redactions
  File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/fitz/fitz.py", line 6610, in annots
    annot = self.load_annot(xref)
  File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/fitz/fitz.py", line 6062, in load_annot
    val.thisown = True
AttributeError: 'SwigPyObject' object has no attribute 'thisown'

If I just took the bit of my code that is doing what your code is doing it would look like:

sensitive = get_sensitive_data(page.get_text("text").split('\n'), sd)
for data in sensitive:
     areas = page.search_for(data)
     [page.add_redact_annot(area, fill=(0, 0, 0)) for area in areas]
page.apply_redactions()

Which works as expected. But, if I used your snippet, what I am trying to do is:

import fitz
doc=fitz.open("PyMuPDF.pdf")
for page in doc:
    for s in samples:
        rl=page.search_for(s)
        for r in rl:
            page.add_redact_annot(r)
        if rl!=[]:
            page.apply_redactions(images=fitz.PDF_REDACT_IMAGE_NONE)

Where samples is a list of string patterns for which to search. Not sure how to apply the last if statement in this example, but hopefully the idea is clear enough.

Thanks again!

[willi3by]

Hopefully to simplify things a bit more, below is the part of the minimal code that is a problem:

for sd in sensitive_data_types:
    sensitive = get_sensitive_data(page.get_text("text").split('\n'), sd)
    for data in sensitive:
        areas = page.search_for(data)
        [page.add_redact_annot(area, fill=(0, 0, 0)) for area in areas]
page.apply_redactions()

I tested with manual iteration:

sd = sensitive_data_types[0]
sensitive = get_sensitive_data(page.get_text("text").split('\n'), sd)
for data in sensitive:
    areas = page.search_for(data)
    [page.add_redact_annot(area, fill=(0, 0, 0)) for area in areas]

sd = sensitive_data_types[1]
sensitive = get_sensitive_data(page.get_text("text").split('\n'), sd)
for data in sensitive:
    areas = page.search_for(data)
    [page.add_redact_annot(area, fill=(0, 0, 0)) for area in areas]

page.apply_redactions()

Which does not throw any error until I try to apply the redactions. I tried looking at the structure of the data before applying the redactions and see no discernible difference among the added redactions so I am not sure why the redactions are successful after the first iteration and fails in subsequent iterations.

Thanks

[JorjMcKie]

Well, again: I can't say anything without having a reproducing file at hand.
All I can do is recommending stuff that may help avoid your obscure condition to arise.
How about this approach:

1. First collect all the rectangles containing sensitive information.
2. Second add all redactions in one loop over all rets
3. Third apply the redactions.

redact_rects = []
redact_rects.extend(page.search_for(some1))
redact_rects.extend(page.search_for(some2))
# ... etc., when finished:
for r in redact_rects:
    page.add_redact_annot(r,...)
page.apply_redactions()

Please be aware, that every execution of page.get_text() and every execution of page.search_for() will create and destroy a TextPage.
If I understand your code correctly, these are several dozens of such things ...

So a variation of your code may be to create one single fitz.TextPage using flags TEXTFLAGS_SEARCH, and use the textpage= parameter in all searches to reuse that same text page. It will remain valid all the way.

[JorjMcKie]

So the above snippet may look like this:

redact_rects = []
tp = page.get_textpage(flags=fitz.TEXTFLAGS_SEARCH)
redact_rects.extend(page.search_for(some1, textpage=tp))
redact_rects.extend(page.search_for(some2, textpage=tp))
# ... etc., when finished:
del tp
for r in redact_rects:
    page.add_redact_annot(r,...)
page.apply_redactions()

[willi3by]

@JorjMcKie of course! Your help is greatly appreciated!

I tried your above suggestion but still get the same error. I also tried taking one of the loops out by using pipes in the regex pattern:

doc = fitz.open(path_to_pdf)
page = doc[0]
sensitive_regex = r"(.*,\s*.*\s.*)|(CFT.*)"
tp = page.get_textpage(flags=fitz.TEXTFLAGS_SEARCH)
sensitive = get_sensitive_data(tp.extractText().split('\n'), sensitive_regex)
for data in sensitive:
     redact_rects.extend(page.search_for(data, textpage=tp))

del tp
for r in redact_rects:
    page.add_redact_annot(r, fill=(0, 0, 0))

page.apply_redactions()

And, it seems that there is something happening at the document level because if I try to extract any page from the original document variable page = doc[0], I get the same error.

Interestingly, when I do not use the OR operator in the regex string, r"(.*,\s*.*\s.*)", the redaction works...

[JorjMcKie]

let's take out this issue to Discussions Q&A ok?
It seems certain to me now, that some error outside PyMuPDF is causing all this.

[JorjMcKie]

Also do not understand why regex is necessary at all, as well as splitting the page text into lines.
If a sensitive thing is present in a line, then it also is present in the full text, or not?

[willi3by]

@JorjMcKie good question. There are multiple sensitive pieces of information contained within the document. Some have a consistent precursor (such as "Acct #: 12345") but some do not (such as the patient name on the document). So I am using regex to find all of the instances of sensitive information that match expected patterns, such as the name is always a character string, followed by a comma and a white space, followed by another character string, followed by another white space, then a single character and a period. Regex was the only way I could think of doing this type of thing. As for splitting the text into lines, it was the only way I could think of to obtain all of the occurrences of a sensitive piece of information on a page. If there is another/better way of approaching this, I am certainly open to suggestions!

[JorjMcKie]

Maybe I am getting it wrong still:
Is your pages organized in lines, such that if a line starts with some standard text indicating sensitivity, then the whole line is redacted?

If this is true, then you really do not need regex. Instead simply extract the text giving you line rectangles.
Then check if a line start indicates sensitivity. If yes, then redact the line.

In this way you even need no searching either!

[willi3by]

Ah I see the confusion. No, unfortunately it is not organized in this way.

[JorjMcKie]

ok, understood your reasoning for using regex, makes sense.
But I still think that using splitlines() is unnecessary: page.get_text(textpage=tp) delivers the complete text of the page in one single string - of course with interspersed \n linebreaks, but that won't harm.

[JorjMcKie]

Could work like this:

redact_rects=[]
for b in page.get_text("dict")["blocks"]:
    for l in b["lines"]:
        bbox = l["bbox"]
        text = " ".join([s["text"] for s in l["spans"]])
        if text.startswith((sensitive1, sensitive2, ...)):
            redact_rects.append(bbox)
for r in redact_rects:
    page.add_redact_annot(r)
page.apply_redactions()

If the issue is that a line must be redacted if it contains critical text, then startswith() must be replaced accordingly, etc.

[JorjMcKie]

If you want, you can send a reprudcing file to my private e-mail [email protected], so it won't become public here.

[willi3by]

I really wish I could but it would be a HIPAA violation :/

[JorjMcKie]

ok, ok, just asking.

[willi3by]

No problem. I completely understand and feel bad that I can't send. I know that makes your job as maintainer a lot harder when trying to troubleshoot

[willi3by]

@JorjMcKie Unfortunately still not working. I even tried creating a file then reloading it with fitz:

doc = fitz.open(path_to_pdf)
page = doc[0]
sensitive = get_sensitive_data(page.get_text().split('\n'), r"(.*,\s*.*\s.*)")
for data in sensitive:
    areas = page.search_for(data)
    [page.add_redact_annot(area, fill=(0, 0, 0)) for area in areas]
page.apply_redactions()
os.remove('tmp_0.pdf')
doc.save('tmp_1.pdf')

Then I iteratively load the newly created pdf and save the next iteration. This actually works perfectly IF there is only one pattern in the regex, i.e. r"(.*,\s*.*\s.*)" instead of r"(.*,\s*.*\s.*)|(Specimen ID:.*)", and IF there is not a loop through the different search patterns. In other words, if I copy paste one pattern at a time, then it works. But as soon as I try multiple or put it in a loop, I get the error.

[JorjMcKie]

...and the exception happens before apply_redactions()?

One more thing you might want to try:
instead of making multiple intermediate saves, etc. try reloading the page with statement page = doc.reload_page(page) exactly like that.
This causes page updates (i.e. new annotations) to land on the page.
Maybe it helps

[willi3by]

Still not working with reload_page.

The exception happens after the first call to apply_redactions. So the first iteration of the loop works perfectly. But after this iteration, anything else I do with the document brings up the error. In other words, after calling page.apply_redactions() once, the command page = doc[0] also throws the error. Additionally, if I try loading the pdf with the first redaction and then try page = doc[0], I get the same error again. If I save as a new file and load doc = fitz.open('temp.pdf'), I can manually do another redaction, but as soon as a loop is involved, I get the error again after the first call to page.apply_redactions().

I am not a coder by trade, so I may be totally off, but it seems that something is being saved in memory within the python kernel as the only way I have found that I can successfully reload and work with the document again is by restarting the kernel.

[willi3by]

Also, if I try to use one call to apply_redactions but with areas derived from multiple regex, I also get the error when I try to call apply_redactions. i.e.,

doc = fitz.open(path_to_pdf)
page = doc[0]
sensitive = get_sensitive_data(page.get_text().split('\n'), r"(.*,\s*.*\s.*)|(Specimen ID:.*)")
for data in sensitive:
    areas = page.search_for(data)
    [page.add_redact_annot(area, fill=(0, 0, 0)) for area in areas]
page.apply_redactions() #This is where the error occurs.

[JorjMcKie]

Well this is like searching a black cat in the night without data at hand, and your reported problems appear just crazy.
So we try some more crazy stuff and replace this

areas = page.search_for(data)
[page.add_redact_annot(area, fill=(0, 0, 0)) for area in areas]

by this:

for area in page.search_for(data):
    page.add_redact_annot(area, fill=0)

[willi3by]

@JorjMcKie That is an excellent analogy. But I have some good news! I think I may be getting to the bottom of the problem. It seems to have something to do with the extracted areas for redaction. As another work around, I made a bash script with the regex patterns as a bash array being passed into a python script to perform the redaction. Doing it this way, I got an extra bit of error message/information:

swig/python detected a memory leak of type 'Annot *', no destructor found.
Traceback (most recent call last):
  File "/Users/willi3by/PycharmProjects/RedactPHIApp/bash_redact.py", line 29, in <module>
    areas = page.search_for(data)
  File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/fitz/utils.py", line 407, in search_for
    tp = page.get_textpage(clip=clip, flags=flags)  # create TextPage
  File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/fitz/fitz.py", line 5584, in get_textpage
    textpage = self._get_textpage(clip, flags=flags, matrix=matrix)
  File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/site-packages/fitz/fitz.py", line 5571, in _get_textpage
    val.thisown = True
AttributeError: 'SwigPyObject' object has no attribute 'thisown'

With the extra bit being swig/python detected a memory leak of type 'Annot *', no destructor found.

I have to stop for today but am investigating the cause and will report back tomorrow.

[willi3by]

@JorjMcKie I got everything to work through iterating with a bash script as opposed to within python. So the script opens and saves the document, closes the python session, then opens a new one to iterate the next regex pattern. I still couldn't track down the root of the problem but it must have something to do with the regex drawing bounding boxes in areas that don't make sense for annotations/redactions. Unfortunately, due to time constraints, I will just have to go with this solution even though it is not optimal. I really appreciate your help!!

[JorjMcKie]

Well, I am pleased you got what you need. Wish I could have helped more, but without an example ...
An afterthought triggered by your side remark:
We might have tried to restrict the TextPage to the document page's rectangle. Your problems might come from a visible page rectangle being smaller than the physical one.
So tp = page.get_textpage(clip=page.rect, flags=...) may have been successful.
But if you are ok now, all is good.

[loganathanspr]

Hi @JorjMcKie : I am facing similar issue. I am able to reproduce the error. Attached the screenshot as well as a pdf file.

Here is the PDF file file:
test.pdf

It's happening in Mac M1 12.6.2, Python 3.8 and Python 3.11 too.

[JorjMcKie]

Cannot reproduce, works without problems in Windows and Linux.

BTW why are you using list(page.annots()), instead directly for annot in page.annots()?

[loganathanspr]

Thanks for the confirmation. Looks like a Mac specific issue. Will try to figure out what the issue is and thanks for flagging list(page.annots()) part.

[loganathanspr]

Just an update: Using the latest PyMuPDF (1.21.2) did not produce the issue. I had the issue with PyMuPDF 1.20.2.
Easy workaround is upgrading to latest version @willi3by @JorjMcKie