Skip to content

Commit

Permalink
removing bug on java function executions
Browse files Browse the repository at this point in the history
  • Loading branch information
ksatzke committed Oct 6, 2020
1 parent 4508ea6 commit e251bf2
Show file tree
Hide file tree
Showing 4 changed files with 75 additions and 5 deletions.
2 changes: 1 addition & 1 deletion ManagementService/python/deployWorkflow.py
Original file line number Diff line number Diff line change
Expand Up @@ -314,7 +314,7 @@ def create_k8s_deployment(email, workflow_info, runtime, gpu_usage, management=F
# apply gpu_usage fraction to k8s deployment configuration
use_gpus = gpu_usage

if not management and use_gpus >= 0:
if not management and use_gpus >= 0 and runtime=="Python":
# overwrite values from values.yaml for new workflows
kservice['spec']['template']['spec']['containers'][0]['resources']['limits']['nvidia.com/gpu'] = str(use_gpus)
kservice['spec']['template']['spec']['containers'][0]['resources']['requests']['nvidia.com/gpu'] = str(use_gpus)
Expand Down
66 changes: 66 additions & 0 deletions Sandbox/Dockerfile_java_gpu
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Copyright 2020 The KNIX Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

#FROM ubuntu:18.04
FROM nvidia/cuda:10.1-cudnn-devel-ubuntu18.04

# Install (as root)
# Base
RUN apt-get update --fix-missing
RUN apt-get -y --no-install-recommends install build-essential
RUN apt-get -y --no-install-recommends install netbase unzip file libmagic1

# Python
RUN apt-get -y --no-install-recommends install python3 python3-dev
RUN apt-get -y --no-install-recommends install python3-pip
RUN apt-get -y --no-install-recommends install zlib1g libssl1.0 libsasl2-2 ca-certificates

RUN /usr/bin/python3 -m pip install --upgrade pip

RUN /usr/bin/python3 -m pip install setuptools
RUN /usr/bin/python3 -m pip install thrift>=0.12.0
RUN /usr/bin/python3 -m pip install anytree
RUN /usr/bin/python3 -m pip install ujsonpath
RUN /usr/bin/python3 -m pip install requests
RUN /usr/bin/python3 -m pip install retry
# remove warnings from anytree package
RUN /usr/bin/python3 -m pip install fastcache
# Needed for multi-language support (currently just Java)
RUN /usr/bin/python3 -m pip install thriftpy2

# Java
RUN apt-get -y --no-install-recommends install openjdk-8-jdk-headless

RUN apt-get -y --no-install-recommends install maven

# Add components (as mfn)
RUN groupadd -o -g 1000 -r mfn && useradd -d /opt/mfn -u 1000 -m -r -g mfn mfn
RUN mkdir /opt/mfn/logs

COPY build/queueservice.jar /opt/mfn/
ADD frontend/frontend /opt/mfn/frontend
ADD build/SandboxAgent.tar.gz /opt/mfn/
ADD build/FunctionWorker.tar.gz /opt/mfn/
ADD build/LoggingService.tar.gz /opt/mfn/

ADD build/JavaRequestHandler.tar.gz /opt/mfn/

RUN chmod +x /opt/mfn/JavaRequestHandler/setup_maven.sh
RUN /opt/mfn/JavaRequestHandler/./setup_maven.sh True
RUN mvn -Duser.home=/tmp -DskipTests -gs /opt/mfn/JavaRequestHandler/maven/sandbox-mvn-settings.xml -f /opt/mfn/JavaRequestHandler/maven/init-mvn.pom.xml dependency:resolve-plugins

RUN chown mfn:mfn -R /opt/mfn
USER mfn
WORKDIR /opt/mfn
CMD ["python3", "/opt/mfn/SandboxAgent/sandboxagent.py"]
6 changes: 5 additions & 1 deletion deploy/helm/microfunctions/templates/management.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,11 @@ data:
{{- /* Disable scale to zero with a minScale of 1. */ -}}
"autoscaling.knative.dev/minScale": "1",
{{- /* Limit scaling to 100 pods. */ -}}
"autoscaling.knative.dev/maxScale": "5"
"autoscaling.knative.dev/maxScale": "5",
{{- /* KubeShare GPU-related configurations */ -}}
"kubeshare/gpu_request": "0.4",
"kubeshare/gpu_limit": "1.0",
"kubeshare/gpu_mem": "3145728000"
},
"labels": {
"app": "microfunctions-workflow",
Expand Down
6 changes: 3 additions & 3 deletions mfn_sdk/mfn_sdk/workflow.py
Original file line number Diff line number Diff line change
Expand Up @@ -102,9 +102,9 @@ def modified(self):

@property
def status(self):
data = self.client.action('getWorkflows',{'workflow':{'id':self.id}})
self._status = data['workflow']['status']
#self._status == "undeployed"
#data = self.client.action('getWorkflows',{'workflow':{'id':self.id}})
#self._status = data['workflow']['status']
self._status == "undeployed"

This comment has been minimized.

Copy link
@iakkus

iakkus Oct 7, 2020

Member

This looks like a bug: it is making a check rather than an assignment.

if self._status == "deployed":
self._endpoints = data['workflow']['endpoints']
else:
Expand Down

1 comment on commit e251bf2

@ksatzke
Copy link
Collaborator Author

@ksatzke ksatzke commented on e251bf2 Oct 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correct, remainder of changes introduced for testing, resolved in subsequent commit

Please sign in to comment.