beam 1215 - upload to ECR #2363

cdhanna · 2023-04-19T20:50:09Z

Ticket

https://disruptorbeam.atlassian.net/browse/BEAM-1215

Brief Description

Moved the upload code into a common directory so it could be shared between Unity and CLI, and used the upload code from the JIRA sample snippet, with some modifications.

Also added a few flags to the beam profile nbomber command that are semi unrelated

Checklist

Have you added appropriate text to the CHANGELOG.md files?
Is there an appropriate JIRA ticket number, and is it named in the title?
Have you documented all your public methods and interfaces? Have you identified intention and assumptions?
Have you included a docs file as /wiki/BEAM-1234.md? You need to provide a docs file.
Does this introduce tech-debt? If so, have you added an entry to the Tech-debt document?

Notes

When you are merging a feature branch into main, please squash merge and make sure the final commit contains any relevent JIRA ticket number. If you are merging from main to staging, or staging to production, please use a regular merge commit.

github-actions · 2023-04-19T21:14:19Z

Have a WebGL build, friend. 9e7dd732df88396c51e6f957b3f78bc6de59e08d

…bleProduct into feature/beam-1215

github-actions · 2023-04-20T17:27:53Z

Have a WebGL build, friend. 6b0a5f36f2fffc0d265eb2209a4ffdf9d8aab821

github-actions · 2023-04-20T17:58:29Z

Have a WebGL build, friend. eaaea73f08543a19ecdb76e1a1cd96a505d2576b

github-actions · 2023-04-20T18:46:39Z

Have a WebGL build, friend. 67395f107318d9787aa818c137f1d0210f6efb68

Leinnan · 2023-04-21T10:32:49Z

cli/cli/Commands/Profiling/RunNBomberCommand.cs

+				}
+				if (!string.IsNullOrEmpty(args.authHeader))
+				{
+					request = request.WithHeader("Authorization", "Bearer 5d938ebe-9cbd-41d6-b077-d8dd89b99819");


Should we include that?

aaaah oh no what have I done.

Leinnan · 2023-04-21T10:35:27Z

microservice/beamable.tooling.common/BeamableLoggerDebug.cs

+namespace Beamable.Server
+{
+   public class BeamableLoggerDebug : IDebug


Maybe we should add at least #pragma warning disable 1591 for that class?

Leinnan · 2023-04-21T10:40:36Z

client/Packages/com.beamable.server/SharedRuntime/ECRUploaderService.cs

+	// public static class Doodad
+	// {
+	// 	[MenuItem("UPLOAD/TESTSTUFF")]
+	// 	public static async void TestTheThing()
+	// 	{
+	// 		var api = BeamEditorContext.Default;
+	// 		await api.InitializePromise;
+	// 		
+	// 		var _client = new HttpClient();
+	// 		var cid = api.Requester.Cid;
+	// 		var pid = api.Requester.Pid;
+	// 		_client.DefaultRequestHeaders.Add("x-ks-clientid", cid);
+	// 		_client.DefaultRequestHeaders.Add("x-ks-projectid", pid);
+	// 		_client.DefaultRequestHeaders.Add("x-ks-token", api.Requester.Token.Token);
+	// 		var serviceUniqueName = "ee6e048a0b6017c4990f13f9d07f9e";//GetHash($"{cid}_{api.ProductionRealm.Pid}_swagger2").Substring(0, 30);
+	//
+	// 		Debug.Log("UNIQUE SERVICE NAME "  + serviceUniqueName);
+	// 		var _uploadBaseUri = $"https://vwe9gi1uxg.execute-api.us-west-2.amazonaws.com/dev/v2/{serviceUniqueName}";
+	//
+	// 		var manifest =
+	// 			"[{\"Config\":\"0028ad28bb75ccb988689b6e5903f2d965d0de3ae5c5572fbc457f8e4542bd1e.json\",\"RepoTags\":null,\"Layers\":[\"1288696addccc4013c5bcf61c1b6c38128a7214a0942976792918b51912d90f7/layer.tar\",\"23747de4211504b6a1554ded518d7abb1e42049460a8eace93cc274553d8e4ea/layer.tar\",\"96c4e657586f01e8c1e113952dbfadddc0036cc8c9f2cc4209233ae2ea4d16cc/layer.tar\",\"a36c795243d77178545508b4f9695c17824d9264bfa9acf4e2e747362ce6260f/layer.tar\",\"80d277b9f6fabe21f1b8432519a55534ba496344ef18c9f5186f368cbb2b49cb/layer.tar\"]}]";
+	//
+	// 		var encodedManifest = Base64Encode(manifest);
+	// 		// var hardcodedBase64 =
+	// 		// 	"W3siQ29uZmlnIjoiMDAyOGFkMjhiYjc1Y2NiOTg4Njg5YjZlNTkwM2YyZDk2NWQwZGUzYWU1YzU1NzJmYmM0NTdmOGU0NTQyYmQxZS5qc29uIiwiUmVwb1RhZ3MiOm51bGwsIkxheWVycyI6WyIxMjg4Njk2YWRkY2NjNDAxM2M1YmNmNjFjMWI2YzM4MTI4YTcyMTRhMDk0Mjk3Njc5MjkxOGI1MTkxMmQ5MGY3L2xheWVyLnRhciIsIjIzNzQ3ZGU0MjExNTA0YjZhMTU1NGRlZDUxOGQ3YWJiMWU0MjA0OTQ2MGE4ZWFjZTkzY2MyNzQ1NTNkOGU0ZWEvbGF5ZXIudGFyIiwiOTZjNGU2NTc1ODZmMDFlOGMxZTExMzk1MmRiZmFkZGRjMDAzNmNjOGM5ZjJjYzQyMDkyMzNhZTJlYTRkMTZjYy9sYXllci50YXIiLCJhMzZjNzk1MjQzZDc3MTc4NTQ1NTA4YjRmOTY5NWMxNzgyNGQ5MjY0YmZhOWFjZjRlMmU3NDczNjJjZTYyNjBmL2xheWVyLnRhciIsIjgwZDI3N2I5ZjZmYWJlMjFmMWI4NDMyNTE5YTU1NTM0YmE0OTYzNDRlZjE4YzlmNTE4NmYzNjhjYmIyYjQ5Y2IvbGF5ZXIudGFyIl19XQo=";
+	// 		//var bytes = UTF8Encoding.ASCII.GetBytes(manifest);
+	// 		// Debug.Log("BYTE SIZE IS " + bytes.Length);//
+	// 		var commitToRegistryResponse = await _client.PostAsync($"{_uploadBaseUri}/submitLayersToECR?imageTag=latest", new StringContent(manifest));
+	// 		
+	// 		try
+	// 		{
+	// 			commitToRegistryResponse.EnsureSuccessStatusCode();
+	// 		}
+	// 		catch (HttpRequestException httpError)
+	// 		{
+	// 			throw new HttpRequestException($"Commiting image to registry failed with: {httpError}");
+	// 		}
+	// 	}
+	// 	
+	// 	public static string Base64Encode(string plainText) 
+	// 	{
+	// 		var plainTextBytes = System.Text.Encoding.UTF8.GetBytes(plainText);
+	// 		return System.Convert.ToBase64String(plainTextBytes);
+	// 	}
+	// 	
+	// 	private readonly static MD5 _md5 = MD5.Create();
+	//
+	// 	static string GetHash(string input)
+	// 	{
+	// 		byte[] data = _md5.ComputeHash(Encoding.UTF8.GetBytes(input));
+	// 		var sBuilder = new StringBuilder();
+	// 		for (int i = 0; i < data.Length; i++)
+	// 		{
+	// 			sBuilder.Append(data[i].ToString("x2"));
+	// 		}
+	// 		return sBuilder.ToString();
+	// 	}
+	//
+	// }


Suggested change

// public static class Doodad

// {

// [MenuItem("UPLOAD/TESTSTUFF")]

// public static async void TestTheThing()

// {

// var api = BeamEditorContext.Default;

// await api.InitializePromise;

//

// var _client = new HttpClient();

// var cid = api.Requester.Cid;

// var pid = api.Requester.Pid;

// _client.DefaultRequestHeaders.Add("x-ks-clientid", cid);

// _client.DefaultRequestHeaders.Add("x-ks-projectid", pid);

// _client.DefaultRequestHeaders.Add("x-ks-token", api.Requester.Token.Token);

// var serviceUniqueName = "ee6e048a0b6017c4990f13f9d07f9e";//GetHash($"{cid}_{api.ProductionRealm.Pid}_swagger2").Substring(0, 30);

//

// Debug.Log("UNIQUE SERVICE NAME " + serviceUniqueName);

// var _uploadBaseUri = $"https://vwe9gi1uxg.execute-api.us-west-2.amazonaws.com/dev/v2/{serviceUniqueName}";

//

// var manifest =

// "[{\"Config\":\"0028ad28bb75ccb988689b6e5903f2d965d0de3ae5c5572fbc457f8e4542bd1e.json\",\"RepoTags\":null,\"Layers\":[\"1288696addccc4013c5bcf61c1b6c38128a7214a0942976792918b51912d90f7/layer.tar\",\"23747de4211504b6a1554ded518d7abb1e42049460a8eace93cc274553d8e4ea/layer.tar\",\"96c4e657586f01e8c1e113952dbfadddc0036cc8c9f2cc4209233ae2ea4d16cc/layer.tar\",\"a36c795243d77178545508b4f9695c17824d9264bfa9acf4e2e747362ce6260f/layer.tar\",\"80d277b9f6fabe21f1b8432519a55534ba496344ef18c9f5186f368cbb2b49cb/layer.tar\"]}]";

//

// var encodedManifest = Base64Encode(manifest);

// // var hardcodedBase64 =

// // "W3siQ29uZmlnIjoiMDAyOGFkMjhiYjc1Y2NiOTg4Njg5YjZlNTkwM2YyZDk2NWQwZGUzYWU1YzU1NzJmYmM0NTdmOGU0NTQyYmQxZS5qc29uIiwiUmVwb1RhZ3MiOm51bGwsIkxheWVycyI6WyIxMjg4Njk2YWRkY2NjNDAxM2M1YmNmNjFjMWI2YzM4MTI4YTcyMTRhMDk0Mjk3Njc5MjkxOGI1MTkxMmQ5MGY3L2xheWVyLnRhciIsIjIzNzQ3ZGU0MjExNTA0YjZhMTU1NGRlZDUxOGQ3YWJiMWU0MjA0OTQ2MGE4ZWFjZTkzY2MyNzQ1NTNkOGU0ZWEvbGF5ZXIudGFyIiwiOTZjNGU2NTc1ODZmMDFlOGMxZTExMzk1MmRiZmFkZGRjMDAzNmNjOGM5ZjJjYzQyMDkyMzNhZTJlYTRkMTZjYy9sYXllci50YXIiLCJhMzZjNzk1MjQzZDc3MTc4NTQ1NTA4YjRmOTY5NWMxNzgyNGQ5MjY0YmZhOWFjZjRlMmU3NDczNjJjZTYyNjBmL2xheWVyLnRhciIsIjgwZDI3N2I5ZjZmYWJlMjFmMWI4NDMyNTE5YTU1NTM0YmE0OTYzNDRlZjE4YzlmNTE4NmYzNjhjYmIyYjQ5Y2IvbGF5ZXIudGFyIl19XQo=";

// //var bytes = UTF8Encoding.ASCII.GetBytes(manifest);

// // Debug.Log("BYTE SIZE IS " + bytes.Length);//

// var commitToRegistryResponse = await _client.PostAsync($"{_uploadBaseUri}/submitLayersToECR?imageTag=latest", new StringContent(manifest));

//

// try

// {

// commitToRegistryResponse.EnsureSuccessStatusCode();

// }

// catch (HttpRequestException httpError)

// {

// throw new HttpRequestException($"Commiting image to registry failed with: {httpError}");

// }

// }

//

// public static string Base64Encode(string plainText)

// {

// var plainTextBytes = System.Text.Encoding.UTF8.GetBytes(plainText);

// return System.Convert.ToBase64String(plainTextBytes);

// }

//

// private readonly static MD5 _md5 = MD5.Create();

//

// static string GetHash(string input)

// {

// byte[] data = _md5.ComputeHash(Encoding.UTF8.GetBytes(input));

// var sBuilder = new StringBuilder();

// for (int i = 0; i < data.Length; i++)

// {

// sBuilder.Append(data[i].ToString("x2"));

// }

// return sBuilder.ToString();

// }

//

// }

Leinnan · 2023-04-21T10:40:54Z

client/Packages/com.beamable.server/SharedRuntime/ECRUploaderService.cs

+		// private async Task<bool> GetLayer(string layer)
+		// {
+		// 	var response = await _client.GetAsync($"{_uploadBaseUri}/doesLayerExist?digest={layer}");
+		//
+		// 	switch (response.StatusCode)
+		// 	{
+		// 		case HttpStatusCode.NotFound:
+		// 			return false; // a valid error code meaning the layer does not exist.
+		// 		case HttpStatusCode.OK:
+		// 			return true; // the body does not need to be serialized, because any 200 implies success.
+		// 		default:
+		// 			response.EnsureSuccessStatusCode();
+		// 			return false;
+		// 	}
+		//
+		// }


Suggested change

// private async Task<bool> GetLayer(string layer)

// {

// var response = await _client.GetAsync($"{_uploadBaseUri}/doesLayerExist?digest={layer}");

//

// switch (response.StatusCode)

// {

// case HttpStatusCode.NotFound:

// return false; // a valid error code meaning the layer does not exist.

// case HttpStatusCode.OK:

// return true; // the body does not need to be serialized, because any 200 implies success.

// default:

// response.EnsureSuccessStatusCode();

// return false;

// }

//

// }

Leinnan

Looks good for the most part, some questions 😅

Leinnan

🐗

cdhanna · 2023-04-21T13:28:17Z

We'll have to wait until the backend changes go live before we can push this.
@ben-beamable , for vis

github-actions · 2023-04-21T13:31:02Z

Have a WebGL build, friend. 3d2988b20ae76320a7a65b68175a978b230cff00

github-actions · 2023-04-21T17:32:17Z

Have a WebGL build, friend. 05a3228cc6167741719165383d3adac05d27bf2c

github-actions · 2023-05-12T14:41:00Z

Have a WebGL build, friend. 173581079c1868518b741dd7fc6896ba9c38a335

# Conflicts: # cli/cli/Commands/Services/ServicesDeployCommand.cs # client/Packages/com.beamable.server/CHANGELOG.md

github-actions · 2023-05-31T19:55:42Z

Have a WebGL build, friend. 8c1b07a59e34d2c9bb6f2852671b65d52acce493

github-actions · 2023-11-20T18:31:36Z

Lightbeam link

github-actions · 2023-11-23T13:11:26Z

Lightbeam link

github-actions · 2023-12-12T14:41:45Z

Lightbeam link

chrisbeamable and others added 2 commits April 19, 2023 16:47

beam 1215

63b72b6

Merge branch 'main' into feature/beam-1215

9c0e43f

chrisbeamable added 4 commits April 20, 2023 13:08

url change

6c85574

Merge branch 'feature/beam-1215' of https://github.com/beamable/Beama…

3674be0

…bleProduct into feature/beam-1215

Merge branch 'main' into feature/beam-1215

67085c6

url fix

3c6a6b6

allow registry override

cefd451

chrisbeamable added 3 commits April 20, 2023 14:21

unity compat

1c3801b

fix test

c733fa9

fixes

3f7d2d1

Leinnan reviewed Apr 21, 2023

View reviewed changes

Leinnan requested changes Apr 21, 2023

View reviewed changes

chrisbeamable added 2 commits April 21, 2023 09:06

pr comments

2076f80

restore

63827e7

Leinnan approved these changes Apr 21, 2023

View reviewed changes

cdhanna added this to the 1.15.0 milestone Apr 21, 2023

partial progress

605d7d1

Merge branch 'main' into feature/beam-1215

975c361

Merge branch 'main' into feature/beam-1215

e8ccfd4

# Conflicts: # cli/cli/Commands/Services/ServicesDeployCommand.cs # client/Packages/com.beamable.server/CHANGELOG.md

Merge branch 'main' into feature/beam-1215

80ba33f

Fix compiler error

5bb8f3c

Merge branch 'main' into feature/beam-1215

0059223

cdhanna removed this from the 1.15.0 milestone Feb 5, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

beam 1215 - upload to ECR #2363

beam 1215 - upload to ECR #2363

cdhanna commented Apr 19, 2023

github-actions bot commented Apr 19, 2023

github-actions bot commented Apr 20, 2023

github-actions bot commented Apr 20, 2023

github-actions bot commented Apr 20, 2023

Leinnan Apr 21, 2023

cdhanna Apr 21, 2023

Leinnan Apr 21, 2023

Leinnan Apr 21, 2023

Leinnan Apr 21, 2023

Leinnan left a comment

Leinnan left a comment

cdhanna commented Apr 21, 2023

github-actions bot commented Apr 21, 2023

github-actions bot commented Apr 21, 2023

github-actions bot commented May 12, 2023

github-actions bot commented May 31, 2023

github-actions bot commented Nov 20, 2023

github-actions bot commented Nov 23, 2023

github-actions bot commented Dec 12, 2023

beam 1215 - upload to ECR #2363

Are you sure you want to change the base?

beam 1215 - upload to ECR #2363

Conversation

cdhanna commented Apr 19, 2023

Ticket

Brief Description

Checklist

Notes

github-actions bot commented Apr 19, 2023

github-actions bot commented Apr 20, 2023

github-actions bot commented Apr 20, 2023

github-actions bot commented Apr 20, 2023

Leinnan Apr 21, 2023

Choose a reason for hiding this comment

cdhanna Apr 21, 2023

Choose a reason for hiding this comment

Leinnan Apr 21, 2023

Choose a reason for hiding this comment

Leinnan Apr 21, 2023

Choose a reason for hiding this comment

Leinnan Apr 21, 2023

Choose a reason for hiding this comment

Leinnan left a comment

Choose a reason for hiding this comment

Leinnan left a comment

Choose a reason for hiding this comment

cdhanna commented Apr 21, 2023

github-actions bot commented Apr 21, 2023

github-actions bot commented Apr 21, 2023

github-actions bot commented May 12, 2023

github-actions bot commented May 31, 2023

github-actions bot commented Nov 20, 2023

github-actions bot commented Nov 23, 2023

github-actions bot commented Dec 12, 2023