Would you please publish a detailed usage (including train my own mask,generate result)in README.md?

[DengHaoyu]

Would you please publish a detailed usage (including train my own mask,generate result)in README.md? I'm just a senior high student ,and it's difficult for me to understand all this.I just wonna know how to use it :-P Thanks.

[breitnerq]

That would be great, i join your request. Also Thanks!

[Ianmcmill]

I am in for this, too.

[TsingCode]

+1

[skylercole]

request bump

[HandsomeDevilv112]

Okay, so, I've spent a number of hours poking and prodding at this, and I've finally got it figured out.
I'm going to assume that you've got it completely installed, and have been able to run the examples using "python gen_all.py"
That part is a spiritual journey that I cannot help you on, as I do not understand how I managed it myself. I'm on ubuntu 18.04, using cuda 10.1. In my effort to get this to work I also installed multiple gcc/g++ , cuda 9, and the most recent version of cmake.
Most of these steps weren't necessary, but I'm practically illiterate, so please just go on this voyage with me.

basic setup:

1. You can remove all files from the data folder, EXCEPT "wikiart_output.txt" I put them into a new folder named "examples".
2. You'll need the basic photoshop or gimp skills required to create the "naive" picture.
3. I'll be using GNU Octave for the dilated mask layer.

Making the initial images.

1. I've managed to skirt this by using an already edited photo of The Undertaker and using gimp to layer it on top of something that interests me.
2. Save the original background image as "0_target.jpg"
3. Save the edited image as "0_naive.jpg"

Making the mask:

1. First turn the background layer black by desaturization/contrast. It's important to use the exact image combination because it's referenced constantly. You can do this through colors>desaturate, then colors>brightness-contrast.
2. Next, do the same to the layer with your new object/favorite professional wrestler. BUT THEN, use colors>invert.
3. Save the combined image as "0_c_mask.png" and place it in the root deep-painterly-harmonization folder.

making the dilated mask:

1. Install, and open gnu octave.
2. cd deep-painterly-harmonization
3. pkg install -forge image
4. pkg load image
   (Failure to do these two steps will give you an fspecial error. I've found I have to run "load image" after I close gnu octave.)
5. run("dilate_mask.m")
6. This will create "0_c_mask_dilated.png"
   Note: I had a helluva time getting this to work.

Next:
Convert "0_c_mask_dilated.png" and "0-c-mask.png" to jpg and place them into the data folder. You should now has five items in the data folder. Target, naive, mask, dilated mask, and wikiart_output.txt

and then~
from the ubuntu command line, run "python gen_all.py" like you did for the examples.

Congratulations, you've now made your own deep-painterly-harmonization. There are ways to streamline this, use the generated png files,

Quick tip.
You might need to adjust the image_size depending on your processing power.

EDIT: I've decided to upload proof that I got it to run... It ain't pretty, but bahgawd, it did the thing.
https://www.facebook.com/pg/Alwood112/photos/?tab=album&album_id=855669681456891

[tombohub]

@HandsomeDevilv112 you did good, too bad I didn't read you post earlier. But, you didnt run('filt_cnn_artifact.m') after gen_all.py is finisnhed. Did you make run('filt_cnn_artifact.m') work??

because it has error

invalid use of script /home/tomo/deep-painterly-harmonization/3rdparty/patchmatch-2.0/nnmex.m in index expression
error: called from
    filt_cnn_artifact at line 50 column 9
    run at line 86 column 5

[HandsomeDevilv112]

I did not experience that, nor do I have any immediate recollection doing anything to that. Doing a quick google investigation, that sure is an octave error.

just to outline my process, I get target, naive, and c_mask all in the root directory. I then run octave in 4 steps:

1. open octave
2. cd deep-painterly-harmonization
3. pkg load image
4. dilate_mask.m

I take the above 3 images and this new one, put them in data, then run "python3 gen_all.py" in terminal. After a few minutes, I get godzilla vs mothra in starry night.

[tombohub]

@HandsomeDevilv112 hehe, yes i do those steps too, but can't finish it with octave command. Do you know where I need to look? I am investigating google like crazy can't figure it out. Trying to compile too, can't make it work.

[HandsomeDevilv112]

I can't say I do. If I stumble upon it, you'll be the first to know, but it's pretty far out of my usual research

[scribblemaniac]

I haven't attempted to work with any of the code in this repository yet, and Matlab/Octave is not a language I have any experience with, but I don't think that you even need it to dilate the mask. Many programs (including GIMP) have a dilate operation. Depending on what exactly it is used for, repeating that operation a bunch of times may be good enough, however the implementation of the dilate operation here is a bit unusual. By studying the code, I think this imagemagick command should produce identical results:

convert 0_c_mask.png -filter gaussian -define filter:support=35 -define filter:sigma=11 -resize 100% -threshold 10% 0_c_mask_dilated.png

@HandsomeDevilv112 Maybe you can confirm that this produces identical, or at least very similar results as the script? If they do not match, can you post an example of a mask and dilated_mask here for reference?

[tombohub]

@scribblemaniac I would say you will get similar results with imagemagick too. I have created diluted mask in photoshop and results are similar.
provided octave srcript is more simpler to use than imagemagick you have there. Just apt install octave. octave run('diluted_mask.m') and thats it

[scribblemaniac]

provided octave srcript is more simpler to use than imagemagick you have there. Just apt install octave. octave run('diluted_mask.m') and thats it

This coming from the person who had issues with it in the first place? You also have to install and load the image package as well it seems. At any rate, I'd say it's simpler because imagemagick comes preinstalled on many systems. A custom script in an less common language is overkill for such a simple operation. People can use whichever method they prefer.

[tombohub]

@cikatomo the issue is with another script...
I was just trying to help. You of course do whatever is best for you

[HandsomeDevilv112]

if you're able to get similar results through imagemagick, and can get similar results through imagemagick...I'll encourage you to use it, because you can get it to work. Once you get those 4 files, all you need is gen_all
I'm not sure which evil entity I sold my soul to that allows me to use octave. Tbh: I'm pretty willing to chalk it up to "dumb luck" and "not knowing when to quit" at this point.

[jjl729756535]

Okay, so, I've spent a number of hours poking and prodding at this, and I've finally got it figured out.
I'm going to assume that you've got it completely installed, and have been able to run the examples using "python gen_all.py"
That part is a spiritual journey that I cannot help you on, as I do not understand how I managed it myself. I'm on ubuntu 18.04, using cuda 10.1. In my effort to get this to work I also installed multiple gcc/g++ , cuda 9, and the most recent version of cmake.
Most of these steps weren't necessary, but I'm practically illiterate, so please just go on this voyage with me.

basic setup:

1. You can remove all files from the data folder, EXCEPT "wikiart_output.txt" I put them into a new folder named "examples".
2. You'll need the basic photoshop or gimp skills required to create the "naive" picture.
3. I'll be using GNU Octave for the dilated mask layer.

Making the initial images.

1. I've managed to skirt this by using an already edited photo of The Undertaker and using gimp to layer it on top of something that interests me.
2. Save the original background image as "0_target.jpg"
3. Save the edited image as "0_naive.jpg"

Making the mask:

1. First turn the background layer black by desaturization/contrast. It's important to use the exact image combination because it's referenced constantly. You can do this through colors>desaturate, then colors>brightness-contrast.
2. Next, do the same to the layer with your new object/favorite professional wrestler. BUT THEN, use colors>invert.
3. Save the combined image as "0_c_mask.png" and place it in the root deep-painterly-harmonization folder.

making the dilated mask:

1. Install, and open gnu octave.
2. cd deep-painterly-harmonization
3. pkg install -forge image
4. pkg load image
   (Failure to do these two steps will give you an fspecial error. I've found I have to run "load image" after I close gnu octave.)
5. run("dilate_mask.m")
6. This will create "0_c_mask_dilated.png"
   Note: I had a helluva time getting this to work.

Next:
Convert "0_c_mask_dilated.png" and "0-c-mask.png" to jpg and place them into the data folder. You should now has five items in the data folder. Target, naive, mask, dilated mask, and wikiart_output.txt

and then~
from the ubuntu command line, run "python gen_all.py" like you did for the examples.

Congratulations, you've now made your own deep-painterly-harmonization. There are ways to streamline this, use the generated png files,

Quick tip.
You might need to adjust the image_size depending on your processing power.

EDIT: I've decided to upload proof that I got it to run... It ain't pretty, but bahgawd, it did the thing.
https://www.facebook.com/pg/Alwood112/photos/?tab=album&album_id=855669681456891

should i modify some source code to adapt the change of images？

[jjl729756535]

@HandsomeDevilv112 you did good, too bad I didn't read you post earlier. But, you didnt run('filt_cnn_artifact.m') after gen_all.py is finisnhed. Did you make run('filt_cnn_artifact.m') work??

because it has error

invalid use of script /home/tomo/deep-painterly-harmonization/3rdparty/patchmatch-2.0/nnmex.m in index expression
error: called from
    filt_cnn_artifact at line 50 column 9
    run at line 86 column 5

some error,did you solved it?

[tombohub]

@jjl729756535 unfortunately, I barely remember anything anymore. I did manage to run it in the end somehow.

Is that step necessary or optional? I played a lot with versions and what not, remind me please

[HandsomeDevilv112]

should i modify some source code to adapt the change of images？
Sorry, I'm not following.
Do whatever you need to do to run gen_all.py, then follow the naming convention of the existing files. I don't recall having to modify code to run gen_all.py initially.

same error, did you solve it?
No. I skipped that step, as I never got it to work, and didn't notice that big of a change from the examples.

[jjl729756535]

@jjl729756535 unfortunately, I barely remember anything anymore. I did manage to run it in the end somehow.

Is that step necessary or optional? I played a lot with versions and what not, remind me please
the step is optional but i want to finish it. i want to know the error was caused by the versions of octave or by invaid use of the fuctions as it mentioned?

[jjl729756535]

should i modify some source code to adapt the change of images？
Sorry, I'm not following.
Do whatever you need to do to run gen_all.py, then follow the naming convention of the existing files. I don't recall having to modify code to run gen_all.py initially.

same error, did you solve it?
No. I skipped that step, as I never got it to work, and didn't notice that big of a change from the examples.

i had run gen_all.py successfully but i stuck in the part of octave.pardon me, i mean if i change the number of picture or the variety of target picture should i modify some source code to adapt that?

[jjl729756535]

by the way ,when i do run('filt_cnn_artifact.m') ,i get
warning:your version of GraphicsMagick limits images to 16 bits per pixel
and then
invalid use of script /deep-painterly-harmonization/3rdparty/patchmatch-2.0/nnmex.m in index expression
error: called from '/deep-painterly-harmonization/filt_cnn_artifact.m' in file /deep-painterly-harmonization/filt_cnn_artifact.m near line 50, column 9
Is there any relationship between the two?

[tombohub]

Did you try to use patchmatch2.1? I see I have it.

Other than that I probably skipped that thing if it's optional

[jjl729756535]

Did you try to use patchmatch2.1? I see I have it.

Other than that I probably skipped that thing if it's optional

I used patchmatch 2.0 which were offered by author,I will try to use 2.1 version and see what will happen,thx!

[jjl729756535]

Did you try to use patchmatch2.1? I see I have it.

Other than that I probably skipped that thing if it's optional

patchmatch2.1 didn't work too ,still error:
invalid use of script /deep-painterly-harmonization/3rdparty/patchmatch-2.1/nnmex.m in index expression
error: called from '/deep-painterly-harmonization/filt_cnn_artifact.m' in file /deep-painterly-harmonization/filt_cnn_artifact.m near line 50, column 9

[tombohub]

I cannot help you anymore unfortunately, I've done it only once, and can't remember anymore

[eyaler]

a working colab: #63