Skip to content

Commit

Permalink
[CELEBORN-1006][FOLLOWUP] Dependency hadoop-client should exclude had…
Browse files Browse the repository at this point in the history
…oop-mapreduce-client dependencies for Hadoop 2

### What changes were proposed in this pull request?

Dependency `hadoop-client` should exclude `hadoop-mapreduce-client` dependencies for Hadoop 2.

### Why are the changes needed?

`hadoop-mapreduce-client` dependencies of `hadoop-client` are unneccessary. Meanwhile, `hadoop-mapreduce-client` dependencies have dependency `javax.servlet:javax.servlet-api` which causes the compilation error of service module after http server refine in 0.5.0 version.

```
$ mvn clean install -Dhadoop.version=2.10.0 -pl service -DskipTests -Dcheckstyle.skip=true -Drat.skip=true -Dspotless.check.skip=true
[INFO] --- scala-maven-plugin:4.7.2:testCompile (scala-test-compile-first)  celeborn-service_2.12 ---
[INFO] Using incremental compilation using Mixed compile order
[INFO] Compiler bridge file: /Users/nicholas/.sbt/1.0/zinc/org.scala-sbt/org.scala-sbt-compiler-bridge_2.12-1.7.1-bin_2.12.18__55.0-1.7.1_20220712T022208.jar
[INFO] compiler plugin: BasicArtifact(com.github.ghik,silencer-plugin_2.12.18,1.7.13,null)
[INFO] compiling 2 Scala sources and 1 Java source to /Users/nicholas/Github/celeborn/service/target/test-classes ...
[ERROR] /Users/nicholas/Github/celeborn/service/src/test/scala/org/apache/celeborn/server/common/http/ApiBaseResourceSuite.scala:33: value readEntity is not a member of javax.ws.rs.core.Response
[ERROR] /Users/nicholas/Github/celeborn/service/src/test/scala/org/apache/celeborn/server/common/http/ApiBaseResourceSuite.scala:77: value readEntity is not a member of javax.ws.rs.core.Response
[ERROR] /Users/nicholas/Github/celeborn/service/src/test/scala/org/apache/celeborn/server/common/http/ApiBaseResourceSuite.scala:84: value readEntity is not a member of javax.ws.rs.core.Response
[ERROR] /Users/nicholas/Github/celeborn/service/src/test/scala/org/apache/celeborn/server/common/http/ApiBaseResourceSuite.scala:91: value readEntity is not a member of javax.ws.rs.core.Response
[ERROR] /Users/nicholas/Github/celeborn/service/src/test/scala/org/apache/celeborn/server/common/http/ApiBaseResourceSuite.scala:97: value readEntity is not a member of javax.ws.rs.core.Response
```

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Manual test.

```
$ mvn clean install -Dhadoop.version=2.10.0 -pl service -DskipTests -Dcheckstyle.skip=true -Drat.skip=true -Dspotless.check.skip=true
[INFO] --- maven-install-plugin:2.5.2:install (default-install)  celeborn-service_2.12 ---
[INFO] Installing /Users/nicholas/Github/celeborn/service/target/celeborn-service_2.12-0.5.0-SNAPSHOT.jar to /Users/nicholas/.m2/repository/org/apache/celeborn/celeborn-service_2.12/0.5.0-SNAPSHOT/celeborn-service_2.12-0.5.0-SNAPSHOT.jar
[INFO] Installing /Users/nicholas/Github/celeborn/service/.flattened-pom.xml to /Users/nicholas/.m2/repository/org/apache/celeborn/celeborn-service_2.12/0.5.0-SNAPSHOT/celeborn-service_2.12-0.5.0-SNAPSHOT.pom
[INFO] Installing /Users/nicholas/Github/celeborn/service/target/celeborn-service_2.12-0.5.0-SNAPSHOT-tests.jar to /Users/nicholas/.m2/repository/org/apache/celeborn/celeborn-service_2.12/0.5.0-SNAPSHOT/celeborn-service_2.12-0.5.0-SNAPSHOT-tests.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  13.928 s
[INFO] Finished at: 2024-05-09T14:59:23+08:00
[INFO] ------------------------------------------------------------------------
```

Closes apache#2497 from SteNicholas/CELEBORN-1006.

Authored-by: SteNicholas <[email protected]>
Signed-off-by: mingji <[email protected]>
(cherry picked from commit a7862c9)
Signed-off-by: mingji <[email protected]>
  • Loading branch information
SteNicholas authored and FMX committed May 9, 2024
1 parent c47fb26 commit f83ab1b
Showing 1 changed file with 12 additions and 0 deletions.
12 changes: 12 additions & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -1041,6 +1041,18 @@
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-annotations</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-app</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-core</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-jobclient</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-yarn-client</artifactId>
Expand Down

0 comments on commit f83ab1b

Please sign in to comment.