Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a warding processor #3

Open
wants to merge 14 commits into
base: master
Choose a base branch
from
Open

Add a warding processor #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
7e49bb5
Add a warding processor
Sep 14, 2016
2dab48f
Update postbuild.sh
howardchung Sep 15, 2016
c7be138
reword annotions and fix code style
Sep 15, 2016
74474b1
Update .travis.yml
howardchung Sep 15, 2016
a72fcaa
extract player_slot for cosmetics
howardchung Sep 18, 2016
47530fb
Merge branch 'master' of github.com:yasp-dota/parser
howardchung Sep 18, 2016
510d6f2
add some owner hero entity data
howardchung Sep 19, 2016
063e10c
update rebuild script
howardchung Sep 20, 2016
39c1428
Add a warding processor
Sep 14, 2016
cb76494
reword annotions and fix code style
Sep 15, 2016
966684a
Merge branch 'master' of github.com:micaelbergeron/parser
Oct 6, 2016
4141fad
correct an edge case on same-tick events
Oct 24, 2016
55a0a45
update the Dockerfile to expose the parser's port
Oct 24, 2016
57fa27b
remove the indirection map (was not used)
Oct 25, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .travis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ notifications:
- [email protected]
- [email protected]
script:
- docker build -t yasp/parser .
- docker build -t odota/parser .
deploy:
provider: script
skip_cleanup: true
Expand Down
2 changes: 2 additions & 0 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,6 @@ WORKDIR /usr/src/parser
ADD . /usr/src/parser
RUN mvn -q -f /usr/src/parser/pom.xml clean install -U

EXPOSE 5600
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how did this work before? port option passed via Docker command line?

Copy link
Author

@micaelbergeron micaelbergeron Oct 25, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is because the new docker-compose is using named services to access the other services instead of going through the host networking. It is good practice to EXPOSE listening ports on your docker components, it makes it easy to expose the service to other components.

As you can see in docker-compose.yml

    environment:
      PARSER_HOST: http://odota-parser:5600
      POSTGRES_URL: postgresql://postgres:postgres@odota-postgres/yasp
      POSTGRES_TEST_URL: postgresql://postgres:postgres@odota-postgres/yasp_test
      READONLY_POSTGRES_URL: postgresql://readonly:readonly@odota-postgres/yasp
      REDIS_URL: redis://odota-redis:6379/0
      REDIS_TEST_URL: redis://odota-redis:6379/1
      CASSANDRA_URL: cassandra://odota-cassandra/yasp
      CASSANDRA_TEST_URL: cassandra://odota-cassandra/yasp_test

I think docker-compose automatically expose the image EXPOSED ports.


CMD ["java", "-jar", "-Xmx256m", "/usr/src/parser/target/stats-0.1.0.jar", "5600"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will you need to update this for the jar name?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I've changed the pom.xml to output the same filename as before.

86 changes: 59 additions & 27 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,27 +40,27 @@
</repository>
</repositories>
<dependencies>
<!--
<dependency>
<groupId>com.github.skadistats</groupId>
<artifactId>clarity</artifactId>
<version>master-SNAPSHOT</version>
</dependency>
-->
<!--
<dependency>
<groupId>com.github.skadistats</groupId>
<artifactId>clarity</artifactId>
<version>70956f5a89fca56f3a292e739ea3eda96a33fd8c</version>
</dependency>
-->
<!--
<dependency>
<groupId>com.github.skadistats</groupId>
<artifactId>clarity</artifactId>
<version>master-SNAPSHOT</version>
</dependency>
-->
<!--
<dependency>
<groupId>com.github.skadistats</groupId>
<artifactId>clarity</artifactId>
<version>70956f5a89fca56f3a292e739ea3eda96a33fd8c</version>
</dependency>
-->

<dependency>
<groupId>com.skadistats</groupId>
<artifactId>clarity</artifactId>
<version>2.1</version>
</dependency>

<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
Expand All @@ -71,24 +71,56 @@
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.2</version>
<artifactId>maven-jar-plugin</artifactId>
<version>3.0.2</version>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<mainClass>yasp.Main</mainClass>
</manifest>
</archive>
<finalName>${project.build.finalName}-original</finalName>
</configuration>
</plugin>
<!-- this is used by clarity-example and clarity-analyzer -->
<plugin>
<groupId>com.jolira</groupId>
<artifactId>onejar-maven-plugin</artifactId>
<version>1.4.4</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
<goal>one-jar</goal>
</goals>
<configuration>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>yasp.Main</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
<configuration>
<filename>${project.build.finalName}.jar</filename>
</configuration>
</plugin>
<!-- for some reason the shade-plugin doesn't work well with clarity processors.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

preferably we should find a way to avoid needing to swap this dependency.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took like 3 hours today trying to make this work. Maybe @spheenik can give some guidance because the shaded jar will crash upon populating the processors. It seems it cannot find the annotations or whatever.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC this produces a jar with a different name. We have to make sure the output file is named the same, or update the Dockerfile to start the correct file.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem here is that there is an annotation scanner in clarity (org.atteo.classindex) that scans annotations on compile time, and produces a file META-INF/annotations/skadistats.clarity.event.Provides which contains a list of all classes with Provides-annotations. If you use shade, you got to make sure this file is present and correctly merged.

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.2</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>yasp.Main</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
-->
</plugins>
</build>
</project>
8 changes: 4 additions & 4 deletions scripts/postbuild.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

if [ -n "$DOCKER_USERNAME" ]; then
docker login -u="$DOCKER_USERNAME" -p="$DOCKER_PASSWORD"
docker tag yasp/parser:latest yasp/parser:${TRAVIS_COMMIT}
docker push yasp/parser:${TRAVIS_COMMIT}
docker push yasp/parser:latest
fi
docker tag odota/parser:latest odota/parser:${TRAVIS_COMMIT}
docker push odota/parser:${TRAVIS_COMMIT}
docker push odota/parser:latest
fi
4 changes: 2 additions & 2 deletions scripts/rebuild.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash

sudo docker build -t odota/parser .
sudo docker rm -fv parser
sudo docker build -t yasp/parser .
sudo docker run -d --name parser --net=host yasp/parser
sudo docker run -d --name parser --net=host odota/parser
0 scripts/test_java_parser.sh → scripts/test.sh
View file
Open in desktop
File renamed without changes.
58 changes: 39 additions & 19 deletions src/main/java/yasp/Main.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,35 +2,55 @@

import java.io.IOException;
import java.io.InputStream;
import java.io.FileInputStream;
import java.io.OutputStream;
import java.net.InetSocketAddress;
import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpHandler;
import com.sun.net.httpserver.HttpServer;

public class Main {

public static void main(String[] args) throws Exception {
HttpServer server = HttpServer.create(new InetSocketAddress(Integer.valueOf(args.length > 0 ? args[0] : "5600")), 0);
server.createContext("/", new MyHandler());
server.setExecutor(java.util.concurrent.Executors.newFixedThreadPool(4));
server.start();
if (args.length > 0 && args[0].equals("--file")) { System.exit(parseFile(args[1])); }
else if (args.length > 0 && args[0].equals("--")) { System.exit(parseStream(System.in, System.out)); }
else { startServer(args); }
}

public static void startServer(String[] args) throws Exception {
HttpServer server = HttpServer.create(new InetSocketAddress(Integer.valueOf(args.length > 0 ? args[0] : "5600")), 0);
server.createContext("/", new MyHandler());
server.setExecutor(java.util.concurrent.Executors.newFixedThreadPool(4));
server.start();
}

private static int parseStream(InputStream is, OutputStream os) throws IOException {
try {
new Parse(is, os);
}
catch (Exception e)
{
e.printStackTrace();
return -1;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extra space

}
finally {
is.close();
os.close();
}

return 0;
}

private static int parseFile(String replayFile) throws Exception {
System.out.print(String.format("Parsing file %s", replayFile));
return parseStream(new FileInputStream(replayFile), System.out);
}

static class MyHandler implements HttpHandler {
@Override
public void handle(HttpExchange t) throws IOException {
t.sendResponseHeaders(200, 0);
InputStream is = t.getRequestBody();
OutputStream os = t.getResponseBody();
try {
new Parse(is, os);
}
catch (Exception e)
{
e.printStackTrace();
}
os.close();
}
@Override
public void handle(HttpExchange t) throws IOException {
t.sendResponseHeaders(200, 0);
parseStream(t.getRequestBody(), t.getResponseBody());
}
}
}
Loading