Use gumtree API in IntelliJ on Windows #275

ztwater · 2022-03-21T11:42:27Z

I have added tree-sitter-parser.py to my system path and it could run from cmd. But when I called the API (e.g. JavaTreeSitterTreeGenerator().generateFrom().file(file); ) from IDE or from commands like "gumtree..." I got this error:

java.io.IOException: Cannot run program "tree-sitter-parser.py" (in directory "C:\Users\24426\AppData\Local\Temp"): CreateProcess error=193, %1 is not a valid Win32 application
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
at com.github.gumtreediff.gen.ExternalProcessTreeGenerator.readStandardOutput(ExternalProcessTreeGenerator.java:35)
at com.github.gumtreediff.gen.treesitter.AbstractTreeSitterGenerator.generate(AbstractTreeSitterGenerator.java:37)
at com.github.gumtreediff.gen.TreeGenerator.generateTree(TreeGenerator.java:41)
at com.github.gumtreediff.gen.TreeGenerator$ReaderConfigurator.reader(TreeGenerator.java:119)
at com.github.gumtreediff.gen.TreeGenerator$ReaderConfigurator.file(TreeGenerator.java:90)
at com.github.gumtreediff.gen.TreeGenerator$ReaderConfigurator.file(TreeGenerator.java:100)

Could you please tell me how to solve this error? I guess processbuilder has something to do with the OS...

The text was updated successfully, but these errors were encountered:

jrfaller · 2022-03-21T12:44:12Z

Hi!

I guess the shebang #!/usr/bin/env python does not work on Windows 😄

However I don't know how to get a Python file running, but if you manage to make a bat file don't hesitate to make a PR!

Cheers.

ztwater · 2022-03-21T13:14:45Z

Thanks a lot! I found that ProcessBuilder can not direct run python scripts, so I changed the arguments of getCommandLine() in AbstractTreeSitterGenerator.java (see here). It is running now but a parse error occurs in TreeIoUtils.java.

while (r.hasNext()) {
    XMLEvent e = r.nextEvent();
    if (e instanceof StartElement) {

javax.xml.stream.XMLStreamException: ParseError at [row,col]:[1,1]
        at java.xml/com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl.next(XMLStreamReaderImpl.java:652)
	at java.xml/com.sun.xml.internal.stream.XMLEventReaderImpl.nextEvent(XMLEventReaderImpl.java:83)
	at com.github.gumtreediff.io.TreeIoUtils$XmlInternalGenerator.generate(TreeIoUtils.java:784)
	at com.github.gumtreediff.gen.TreeGenerator.generateTree(TreeGenerator.java:41)
        ...

jrfaller · 2022-03-21T14:06:17Z

Can you print the XML being parser to see what the problem is?

ztwater · 2022-03-21T18:12:08Z

This is the XML tree (from a java code snippet on GitHub) that generated by TreeGenerator:

<?xml version="1.0" ?>
<tree type="program" pos="0" length="598">
	<tree type="type_declaration" pos="0" length="598">
		<tree type="modifiers" pos="0" length="6">
			<tree type="visibility" pos="0" length="6" label="public"/>
		</tree>
		<tree type="type_keyword" pos="7" length="5" label="class"/>
		<tree type="identifier" pos="13" length="3" label="foo"/>
		<tree type="type_body" pos="16" length="582">
			<tree type="method_declaration" pos="18" length="578">
				<tree type="modifiers" pos="18" length="19">
					<tree type="visibility" pos="18" length="6" label="public"/>
					<tree type="final" pos="25" length="5" label="final"/>
					<tree type="static" pos="31" length="6" label="static"/>
				</tree>
				<tree type="type" pos="38" length="6" label="String"/>
				<tree type="identifier" pos="45" length="7" label="uniplus"/>
				<tree type="formal_parameters" pos="52" length="10">
					<tree type="formal_parameter" pos="53" length="8">
						<tree type="type" pos="53" length="6" label="String"/>
						<tree type="identifier" pos="60" length="1" label="s"/>
					</tree>
				</tree>
				<tree type="block" pos="63" length="533">
					<tree type="if_statement" pos="69" length="47">
						<tree type="if" pos="69" length="2" label="if"/>
						<tree type="parenthesized_expression" pos="72" length="17">
							<tree type="binary_expression" pos="73" length="15">
								<tree type="method_invocation" pos="73" length="10">
									<tree type="identifier" pos="73" length="1" label="s"/>
									<tree type="identifier" pos="75" length="6" label="length"/>
									<tree type="argument_list" pos="81" length="2"/>
								</tree>
								<tree type="comparison_operator" pos="84" length="2" label="=="/>
								<tree type="decimal_integer_literal" pos="87" length="1" label="0"/>
							</tree>
						</tree>
						<tree type="block" pos="90" length="26">
							<tree type="return_statement" pos="100" length="10">
								<tree type="string_literal" pos="107" length="2" label="&quot;&quot;"/>
							</tree>
						</tree>
					</tree>
					<tree type="block_comment" pos="121" length="55" label="/* This is just the minimum; sb will grow as needed. */"/>
					<tree type="local_variable_declaration" pos="181" length="55">
						<tree type="type" pos="181" length="12" label="StringBuffer"/>
						<tree type="variable_declarator" pos="194" length="41">
							<tree type="identifier" pos="194" length="2" label="sb"/>
							<tree type="affectation_operator" pos="197" length="1" label="="/>
							<tree type="object_creation_expression" pos="199" length="36">
								<tree type="new" pos="199" length="3" label="new"/>
								<tree type="type" pos="203" length="12" label="StringBuffer"/>
								<tree type="argument_list" pos="215" length="20">
									<tree type="binary_expression" pos="216" length="18">
										<tree type="decimal_integer_literal" pos="216" length="1" label="2"/>
										<tree type="arithmetic_operator" pos="218" length="1" label="+"/>
										<tree type="binary_expression" pos="220" length="14">
											<tree type="decimal_integer_literal" pos="220" length="1" label="3"/>
											<tree type="arithmetic_operator" pos="222" length="1" label="*"/>
											<tree type="method_invocation" pos="224" length="10">
												<tree type="identifier" pos="224" length="1" label="s"/>
												<tree type="identifier" pos="226" length="6" label="length"/>
												<tree type="argument_list" pos="232" length="2"/>
											</tree>
										</tree>
									</tree>
								</tree>
							</tree>
						</tree>
					</tree>
					<tree type="expression_statement" pos="241" length="16">
						<tree type="method_invocation" pos="241" length="15">
							<tree type="identifier" pos="241" length="2" label="sb"/>
							<tree type="identifier" pos="244" length="6" label="append"/>
							<tree type="argument_list" pos="250" length="6">
								<tree type="string_literal" pos="251" length="4" label="&quot;U+&quot;"/>
							</tree>
						</tree>
					</tree>
					<tree type="for_statement" pos="262" length="306">
						<tree type="local_variable_declaration" pos="267" length="10">
							<tree type="type" pos="267" length="3" label="int"/>
							<tree type="variable_declarator" pos="271" length="5">
								<tree type="identifier" pos="271" length="1" label="i"/>
								<tree type="affectation_operator" pos="273" length="1" label="="/>
								<tree type="decimal_integer_literal" pos="275" length="1" label="0"/>
							</tree>
						</tree>
						<tree type="binary_expression" pos="278" length="14">
							<tree type="identifier" pos="278" length="1" label="i"/>
							<tree type="comparison_operator" pos="280" length="1" label="&lt;"/>
							<tree type="method_invocation" pos="282" length="10">
								<tree type="identifier" pos="282" length="1" label="s"/>
								<tree type="identifier" pos="284" length="6" label="length"/>
								<tree type="argument_list" pos="290" length="2"/>
							</tree>
						</tree>
						<tree type="update_expression" pos="294" length="3">
							<tree type="identifier" pos="294" length="1" label="i"/>
							<tree type="increment_operator" pos="295" length="2" label="++"/>
						</tree>
						<tree type="block" pos="299" length="269">
							<tree type="expression_statement" pos="309" length="49">
								<tree type="method_invocation" pos="309" length="48">
									<tree type="identifier" pos="309" length="2" label="sb"/>
									<tree type="identifier" pos="312" length="6" label="append"/>
									<tree type="argument_list" pos="318" length="39">
										<tree type="method_invocation" pos="319" length="37">
											<tree type="identifier" pos="319" length="6" label="String"/>
											<tree type="identifier" pos="326" length="6" label="format"/>
											<tree type="argument_list" pos="332" length="24">
												<tree type="string_literal" pos="333" length="4" label="&quot;%X&quot;"/>
												<tree type="," pos="337" length="1" label=","/>
												<tree type="method_invocation" pos="339" length="16">
													<tree type="identifier" pos="339" length="1" label="s"/>
													<tree type="identifier" pos="341" length="11" label="codePointAt"/>
													<tree type="argument_list" pos="352" length="3">
														<tree type="identifier" pos="353" length="1" label="i"/>
													</tree>
												</tree>
											</tree>
										</tree>
									</tree>
								</tree>
							</tree>
							<tree type="if_statement" pos="367" length="125">
								<tree type="if" pos="367" length="2" label="if"/>
								<tree type="parenthesized_expression" pos="370" length="40">
									<tree type="binary_expression" pos="371" length="38">
										<tree type="method_invocation" pos="371" length="16">
											<tree type="identifier" pos="371" length="1" label="s"/>
											<tree type="identifier" pos="373" length="11" label="codePointAt"/>
											<tree type="argument_list" pos="384" length="3">
												<tree type="identifier" pos="385" length="1" label="i"/>
											</tree>
										</tree>
										<tree type="comparison_operator" pos="388" length="1" label="&gt;"/>
										<tree type="field_access" pos="390" length="19">
											<tree type="identifier" pos="390" length="9" label="Character"/>
											<tree type="identifier" pos="400" length="9" label="MAX_VALUE"/>
										</tree>
									</tree>
								</tree>
								<tree type="block" pos="411" length="81">
									<tree type="expression_statement" pos="425" length="4">
										<tree type="update_expression" pos="425" length="3">
											<tree type="identifier" pos="425" length="1" label="i"/>
											<tree type="increment_operator" pos="426" length="2" label="++"/>
										</tree>
									</tree>
									<tree type="block_comment" pos="430" length="52" label="/****WE HATES UTF-16! WE HATES IT FOREVERSES!!!****/"/>
								</tree>
							</tree>
							<tree type="if_statement" pos="501" length="61">
								<tree type="if" pos="501" length="2" label="if"/>
								<tree type="parenthesized_expression" pos="504" length="18">
									<tree type="binary_expression" pos="505" length="16">
										<tree type="binary_expression" pos="505" length="3">
											<tree type="identifier" pos="505" length="1" label="i"/>
											<tree type="arithmetic_operator" pos="506" length="1" label="+"/>
											<tree type="decimal_integer_literal" pos="507" length="1" label="1"/>
										</tree>
										<tree type="comparison_operator" pos="509" length="1" label="&lt;"/>
										<tree type="method_invocation" pos="511" length="10">
											<tree type="identifier" pos="511" length="1" label="s"/>
											<tree type="identifier" pos="513" length="6" label="length"/>
											<tree type="argument_list" pos="519" length="2"/>
										</tree>
									</tree>
								</tree>
								<tree type="block" pos="523" length="39">
									<tree type="expression_statement" pos="537" length="15">
										<tree type="method_invocation" pos="537" length="14">
											<tree type="identifier" pos="537" length="2" label="sb"/>
											<tree type="identifier" pos="540" length="6" label="append"/>
											<tree type="argument_list" pos="546" length="5">
												<tree type="string_literal" pos="547" length="3" label="&quot;.&quot;"/>
											</tree>
										</tree>
									</tree>
								</tree>
							</tree>
						</tree>
					</tree>
					<tree type="return_statement" pos="573" length="21">
						<tree type="method_invocation" pos="580" length="13">
							<tree type="identifier" pos="580" length="2" label="sb"/>
							<tree type="identifier" pos="583" length="8" label="toString"/>
							<tree type="argument_list" pos="591" length="2"/>
						</tree>
					</tree>
				</tree>
			</tree>
		</tree>
	</tree>
</tree>

When I looked into the XMLEventReader in TreeIoUtils.java, I got <?xml version="null" encoding='null' standalone='no'?> as the first element. Strangely, r.hasNext() returns true while I got parse error when I run nextEvent().

jrfaller · 2022-03-22T14:57:33Z

OK strange, the XML your parsed seems correct

jrfaller self-assigned this Mar 21, 2022

jrfaller added the bug label Mar 21, 2022

jrfaller added the help wanted label Mar 21, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use gumtree API in IntelliJ on Windows #275

Use gumtree API in IntelliJ on Windows #275

ztwater commented Mar 21, 2022

jrfaller commented Mar 21, 2022

ztwater commented Mar 21, 2022

jrfaller commented Mar 21, 2022

ztwater commented Mar 21, 2022

jrfaller commented Mar 22, 2022

Use gumtree API in IntelliJ on Windows #275

Use gumtree API in IntelliJ on Windows #275

Comments

ztwater commented Mar 21, 2022

jrfaller commented Mar 21, 2022

ztwater commented Mar 21, 2022

jrfaller commented Mar 21, 2022

ztwater commented Mar 21, 2022

jrfaller commented Mar 22, 2022