How can I manually alter notebook tables?

[carlosparadis]

@tuejari

As mentioned in the smells showcase, it is strongly encouraged to manually inspect the assigned identity to the various users and I understand why. But to me, it is not 100% clear how to inspect these and how to fix them. I found multiple distinct identity ids which seem to refer to the same person. How can I manually alter this data?

[carlosparadis]

kaiaulu/vignettes/social_smell_showcase.Rmd

Lines 164 to 175 in 6e35597

	```{r}
	#Identity matching
	project_log <- list(project_git=project_git,project_mbox=project_mbox)
	project_log <- identity_match(project_log,
	name_column = c("author_name_email","reply_from"),
	assign_exact_identity,
	use_name_only=TRUE,
	label = "raw_name")
	project_git <- project_log[["project_git"]]
	project_mbox <- project_log[["project_mbox"]]
	```

After you run this code block, you should have in project_git and project_mbox one data.table in each. Each column is just text, so you can manually select the rows you wish and edit the text directly in identity_id, removing unnecessary rows, etc as you would in any R object.

You can use R Studio to run up to and before this code block, then this code block, then run the remainder of the code blocks. There shouldn't be any need to export the table outside to other program: You can do it directly on the console. You can also open another blank notebook to paste and edit code if that is easier since the different tabs all share the same R session.

This question is precisely why I chose the function library + notebooks architecture for Kaiaulu rather than all-in-all-out. I looked at the cases you sent in private via e-mail (and omitted here for privacy of the original author for when the repo goes public), and I can't think of a heuristic that could capture it as easily as a person would, although it did detect some of the authors in question identities, which saves some manual work. However, if you can think of one, or find cases where you can, please feel free to open an issue so I can add it to Kaiaulu (please do edit the actual name and email so only the pattern is kept but not the actual identity).

p.s.: The various heuristics Kaiaulu uses can be found here as unit tests: https://github.com/sailuh/kaiaulu/blob/master/tests/testthat/test-identity.R. All names and e-mails there have been edited from the original, just the pattern was kept.

Let me know if I misunderstood the question!

[tuejari]

You can use R Studio to run up to and before this code block, then this code block, then run the remainder of the code blocks.

This is what I have been doing so far.

There shouldn't be any need to export the table outside to other program: You can do it directly on the console.

This is what I am not familiar with. I do not have that much experience with R and I don't know how to edit these data tables in the console. Using RStudio's Environment I managed to view the data.table (View(project_git) and View(project_mbox)), but I cannot edit the data.tables that way. That's why I was trying to export and import it in the environment.

[carlosparadis]

I understand the point of confusion now. Have you used Python interactively before? (Open command line + type "python"?). If yes, then you can also do the same with R installed for a console: You launch the command line, and then type "R" on a mac to launch it. This is the screen that appears on the bottom of RStudio on the "Console" tab.

Meanwhile, the pane where you see the code is equivalent to a text editor. So RStudio itself is just putting these two together for you, but you could have the equivalent by opening on a command line and text editor.

When you click "Run all":

RStudio just conveniently copy and paste each code block in the "Console" pane below (you can actually see the text appearing of the code blocks in the console if you scroll around after you run).

But again, this would be the same if you had a text editor and your command line prompt open and copy and pasted yourself the code.

In the command line, the variables stay loaded (there are some functions in R, Python, etc that can show but it's cumbersome). An inconvenience of using the command line alone is that you don't easily see what variables are loaded. This is what the right pane is for on the top right of Rstudio:

So you can quickly check here what variables you have. This is what I was referring to: You can manipulate the table directly on memory here. Table manipulation in R is very close in concept to the Pandas library in Python (you reason data.frames, columns, rows, rather than Python's data objects of lists, tuples, dictionaries).

I hope this gives you an idea of what I was suggesting. Languages like R, Octave, and Matlab which are data-oriented (or Python with Pandas and Jupyter Notebooks) give you this "on the fly" environment so you can experiment with the data as you build code, rather than for example, C where you have to save all files, compile and then run the final file as a binary taking input and output.

The View() function you refer to is indeed an RStudio function, but it is just a prettier version of copying the project_git variable and pasting in the console (you should give it a try).

Say you want to edit the first row of project git on identity_id column, you could do:

project_git[1]$identity_id <- "some string that is preferred". That replaces the first row in said column. With a few lines you can also delete rows, add rows, etc. Since Kaiaulu uses data.tables instead of data.frames, you can see the data.table package on how to do these operations (they're done all over the notebook code too):

https://rdatatable.gitlab.io/data.table/

To be clear, you edit the tables programming in R, not using a graphic interface.

[tuejari]

Have you used Python interactively before?

I have used Python almost exclusively for scripts, but I do know about Python's interactive mode.

So you can quickly check here what variables you have. This is what I was referring to: You can manipulate the table directly on memory here. Table manipulation in R is very close in concept to the Pandas library in Python (you reason data.frames, columns, rows, rather than Python's data objects of lists, tuples, dictionaries).

I hope this gives you an idea of what I was suggesting.

To be clear, you edit the tables programming in R, not using a graphic interface.

Yes, I was really just thinking in the wrong direction since I was trying to edit the tables using the graphic interface (e.g. through RStudio's environment). It also made sense to me to edit such a large table externally for convenience and then import it again in the environment.

Thank you for the detailed explanation!