Can't create FargateService
polkx opened this issue · 12 comments
When I try to create a FargateServiceArgs class the Error shows up and tells me that I must add taskDefinition or taskDefinitionArgs. Even when I add taskDefinition, error still shows up. Example:
Dependencies:
//> using scala "3.3.1"
//> using options -Werror -Wunused:all -Wvalue-discard -Wnonunit-statement
//> using plugin "org.virtuslab::besom-compiler-plugin:0.2.1"
//> using dep "org.virtuslab::besom-core:0.2.1"
//> using dep "org.virtuslab::besom-awsx:2.5.0-core.0.2"Main class:
import besom.*
import besom.api.aws.ecs.Cluster
import besom.api.awsx.ecs.{FargateService, FargateServiceArgs}
import besom.api.awsx.ecs.inputs.{
FargateServiceTaskDefinitionArgs,
TaskDefinitionContainerDefinitionArgs,
TaskDefinitionPortMappingArgs
}
@main def main = Pulumi.run {
val cluster = Cluster("cluster")
val taskDefinition =
FargateServiceTaskDefinitionArgs(
container = TaskDefinitionContainerDefinitionArgs(
name = "my-task-def",
image = "nginx",
memory = 512,
portMappings = List(TaskDefinitionPortMappingArgs(containerPort = 80))
)
)
val service = FargateService(
"my-service",
FargateServiceArgs(
cluster = cluster.arn,
// taskDefinition = "my-task-def",
taskDefinitionArgs = taskDefinition,
desiredCount = 1
)
)
Stack.exports(url = service.urn)
}Stack trace:
Previewing update (dev):
Type Name Plan Info
+ pulumi:pulumi:Stack aws-hello-fargate-dev create 2 errors; 37 messages
+ └─ awsx:ecs:FargateService my-service create
Diagnostics:
pulumi:pulumi:Stack (aws-hello-fargate-dev):
2024.02.21 11:44:24:850 scala-execution-context-global-20 [resource: my-service[awsx:ecs:FargateService]] ERROR besom.internal.ResourceDecoder.resolve:91
Resolve resource: received error from gRPC call: 'UNKNOWN: Either `taskDefinition` or `taskDefinitionArgs` must be provided.', failing resolution
2024.02.21 11:44:24:877 main ERROR besom.internal.BesomModule.run:69
io.grpc.StatusRuntimeException: UNKNOWN: Either `taskDefinition` or `taskDefinitionArgs` must be provided.
at io.grpc.Status.asRuntimeException(Status.java:537)
at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:538)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:567)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:71)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:735)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:716)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.lang.Thread.run(Thread.java:1583)
Exception in thread "main" io.grpc.StatusRuntimeException: UNKNOWN: Either `taskDefinition` or `taskDefinitionArgs` must be provided.
at io.grpc.Status.asRuntimeException(Status.java:537)
at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:538)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:567)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:71)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:735)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:716)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
Error: Either `taskDefinition` or `taskDefinitionArgs` must be provided.: Error: Either `taskDefinition` or `taskDefinitionArgs` must be provided.
at new FargateService (/snapshot/awsx/bin/ecs/fargateService.js:47:19)
at awsx:ecs:FargateService (/snapshot/awsx/bin/resources.js:25:45)
at construct (/snapshot/awsx/bin/resources.js:43:12)
at Provider.construct (/snapshot/awsx/bin/index.js:39:52)
at Server.<anonymous> (/snapshot/awsx/node_modules/@pulumi/pulumi/provider/server.js:315:52)
at Generator.next (<anonymous>)
at fulfilled (/snapshot/awsx/node_modules/@pulumi/pulumi/provider/server.js:18:58)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
error: besom.internal.ResourceDecoder.resolve:91 [resource: my-service[awsx:ecs:FargateService]] Resolve resource: received error from gRPC call: 'UNKNOWN: Either `taskDefinition` or `taskDefinitionArgs` must be provided.', failing resolution
error: an unhandled error occurred: '/opt/homebrew/bin/scala-cli /opt/homebrew/bin/scala-cli run .' exited with non-zero exit code: 1
Another example, the same result:
import besom.*
import besom.api.awsx.ecs.FargateService
@main def main = Pulumi.run {
val service = FargateService("my-service")
Stack.exports(url = service.urn)
}Thank you. We need to investigate why is this happening.
Note to self: the relevant code in the provider: https://github.com/pulumi/pulumi-awsx/blob/5586b5776d22c437044f2e20b23a1768d786fb97/awsx/ecs/ec2Service.ts#L53
Glad I found this issue!
I have a service definition like this:
//> using scala "3.3.3"
//> using options -Werror -Wunused:all -Wvalue-discard -Wnonunit-statement
//> using dep "org.virtuslab::besom-core:0.2.2"
//> using dep "org.virtuslab::besom-aws:6.23.0-core.0.2"
//> using dep "org.virtuslab::besom-awsx:2.5.0-core.0.2"
//> using dep "com.lihaoyi::upickle::3.2.0"
import besom.*
import besom.api.aws, besom.api.awsx, awsx.ecr, awsx.lb, awsx.ecs
import besom.api.awsx.ecs.inputs.*
import besom.api.awsx.lb.ApplicationLoadBalancerArgs
import besom.api.aws.ecs.inputs.ServiceNetworkConfigurationArgs
import besom.api.aws.cloudwatch.LogGroupArgs
import besom.json.*
@main def main = Pulumi.run {
val repository =
ecr.Repository(
"sentiment-service-repo",
ecr.RepositoryArgs(forceDelete = true)
)
val image = ecr.Image(
"sentiment-service-image",
ecr.ImageArgs(
repositoryUrl = repository.url,
context = p"../app",
platform = "linux/amd64"
)
)
val vpc = awsx.ec2.Vpc("sentiment-service-vpc")
val loadBalancer = lb.ApplicationLoadBalancer(
"sentiment-service-lb",
ApplicationLoadBalancerArgs(subnetIds = vpc.publicSubnetIds)
)
val cluster = aws.ecs.Cluster("sentiment-service-cluster")
val logGroup =
aws.cloudwatch.LogGroup(
"sentiment-service-log-group",
LogGroupArgs(retentionInDays = 7, name = "sentiment-service-logs")
)
val service =
image.imageUri.flatMap: image =>
ecs.FargateService(
"sentiment-service-fargate",
ecs.FargateServiceArgs(
networkConfiguration = ServiceNetworkConfigurationArgs(
subnets = vpc.publicSubnetIds,
assignPublicIp = true,
securityGroups =
loadBalancer.defaultSecurityGroup.map(_.map(_.id)).map(_.toList)
),
cluster = cluster.arn,
taskDefinitionArgs = FargateServiceTaskDefinitionArgs(
containers = Map(
"sentiment-service" -> TaskDefinitionContainerDefinitionArgs(
image = image,
name = "sentiment-service",
cpu = 128,
memory = 512,
essential = true,
logConfiguration = TaskDefinitionLogConfigurationArgs(
logDriver = "awslogs",
options = JsObject(
"awslogs-group" -> JsString("log-group"),
"awslogs-region" -> JsString("us-east-1"),
"awslogs-stream-prefix" -> JsString("ecs")
)
),
/*portMappings = List(
TaskDefinitionPortMappingArgs(
targetGroup = loadBalancer.defaultTargetGroup
)
)*/
)
)
)
)
)
Stack(logGroup).exports(
image = image.imageUri,
service = service,
vpc = vpc.id,
cluster = cluster.id,
url = p"http://${loadBalancer.loadBalancer.dnsName}"
)
}And I get this exception IF I do any of the steps below:
Diagnostics:
pulumi:pulumi:Stack (besom-smithy4s-kiss-dev):
Error: Exactly one of [container] or [containers] must be provided: Error: Exactly one of [container] or [containers] must be provided
at normalizeTaskDefinitionContainers (/snapshot/awsx/bin/ecs/containers.js:37:15)
at new FargateTaskDefinition (/snapshot/awsx/bin/ecs/fargateTaskDefinition.js:34:79)
at new FargateService (/snapshot/awsx/bin/ecs/fargateService.js:40:30)
at awsx:ecs:FargateService (/snapshot/awsx/bin/resources.js:25:45)
at construct (/snapshot/awsx/bin/resources.js:43:12)
at Provider.construct (/snapshot/awsx/bin/index.js:39:52)
at Server.<anonymous> (/snapshot/awsx/node_modules/@pulumi/pulumi/provider/server.js:315:52)
at Generator.next (<anonymous>)
at fulfilled (/snapshot/awsx/node_modules/@pulumi/pulumi/provider/server.js:18:58)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
Here are the things that trigger this error:
- Uncommenting the
portMappingblock - Not using
flatMapwith imageUri, and instead referencing it directly
Now, I've only been using Pulumi for like 2 days, so I don't know if it's a semantics problem or what, but the exception is very confusing.
Note that there's a pulumi-java issue: pulumi/pulumi-java#1325 and this awsx one pulumi/pulumi-awsx#820 aaaand I'm only now noticing that @pawelprazak is aware :D
IIRC I've tried a couple of workarounds from both issues but neither of them worked with Besom.
Hey, I think we have this fixed on current main (with some workarounds for upstream issues). We will cut a release soon. @pawelprazak can you take a look on this and advise if anything can be done to work around this issue on 0.2.2? Paweł has made a big refactoring in serde layer that brought us to be completely up to date with upstream Pulumi serde and this was done to fix this issue mostly. This change is however binary incompatible on core-providers edge so we have to release it as a part of 0.3.x. I think we should do this really fast, considering I found a nasty grpc memoization issue when writing post-Scalar blogpost (also already fixed).
Hi @keynmol, thank you for the reproduction and I'm sorry you've stumbled upon this cursed issue :)
What we know:
awsxprovider (technically it is a multi-language component) has a known issue when used from SDKs other than TS/nodejs, unfortunately I was not able to get the to root cause upstream yet- there should be a way to work around those issues (at least the ones I know of) (https://github.com/polkx/besom-bootzooka/blob/main/aws-fargate-bootzooka/Main.scala#L137)
- there is a new-ish serialization framework used underneath by Pulumi, and it was ported to besom, awaiting release (#414), it should help, but is not required for the solution
The key workaround was to use explicit val task = awsx.ecs.FargateTaskDefinition(...) instead of the one inlined in the arguments. Please let me know if the issue persists after this change, and I'll try to help to find more workarounds if necessary.
The workarounds might work on the current 0.2.2, but unfortunately there are known issues that required binary compatibility breaking changes and are awaiting for 0.3.0, that we plan to release soon.
P.s. I'm very curious if the flatMap will no longer be necessary with the workaround described above, because if not I would worry me ;)
Good to hear 👍 I'm doing this for a blogpost as well so can easily wait.
I will try the workaround, and will excitedly wait for the 0.3.0 :)
Actually it seems I was able to achieve what I wanted (I think, the architecture looks and works the way I wanted) by
- removing target group from port mapping, only keeping containerPort
- adding loadBalancers block to FargateService:
.. 60 loadBalancers = List(
.. 61 ServiceLoadBalancerArgs(
.. 62 containerName = "sentiment-service",
.. 63 containerPort = 80,
.. 64 targetGroupArn = loadBalancer.defaultTargetGroup.arn
.. 65 )
.. 66 ),Btw you should not need to flatMap on imageUri property to use its value as an argument for another Args. Args are built in a way that allows them to consume both raw values and values wrapped in Outputs (opaque type Input[A] = A | Output[A] essentially). We are working on adding a warning when you call a resource constructor in a body of lambda passed to flatMap on Output because, and this is a problem well understood in Scala community thanks to CT discourse, it is impossible for Pulumi to form a static plan of deployment when dynamic, monadic APIs are used (so, basically, applicative vs monadic from mill vs sbt discourse). In dry run computed properties (Outputs on resources) are unknown and behave like Option None - they short circuit and do not run flatMaps. Therefore they cut out subtrees of the deployment plan and that's exactly why we don't really flatMap anywhere beside computing derived properties that we later feed to args of another resource constructor that is called statically, not in a flatMap.
That's a gotcha coming from the elasticity of Pulumi that Scala's ease of use of flatMap sort of exacerbates. Compile warn should help push people to use this power only when really necessary.
Glad to hear you were able to workaround the issues. Feel free to reach out if any more problems crop up. I'll leave this open until release of 0.3.0, but I'm afraid that the underlying issue appears to be upstream unfortunately.
@lbialy WDYT, can we close this now that the 0.3.x was released? I think we've done what we could on our side and the main problem in this thread is upstream, in the awsx provider.
It is an open problem that is not fixed. I think we should leave this as a lower priority bug report and check after a new release of pulumi/pulumi-eks (there has been a new one few hours ago - 2.5.2 btw but I don't expect it to fix this problem).