This Jenkins plugin uses AWS Lambda to host jobs execution.
Jenkins delegates to AWS Lambdas the execution of the builds on Lambda based agents (runtimes). Each Jenkins build is executed on a Lambda execution that is released at the end of the build.
- use GitHub Issues to report issues / feature requests
Given the limitations of the AWS Lambda engine :
- jobs can't exceed a duration of 15 minutes
- jobs can't use more than 512 MB of storage
- Jenkins with at least version 1.635 (beware of using Lambda agents with the remoting library compatible with your jenkins)
- an AWS Account
IMPORTANT : Jenkins will by default mark as unavailable any agent with low space on disk or low space in temp. Given the limited space of 512MB available in the writable temp filesystem of a Lambda, you must adapt or disable this behavior under the "Nodes Configuration" section of your Jenkins (url: /computer/configure
).
Use the Jenkins plugin manager to install the AWS Lambda Cloud
See this repository on how to build and deploy Lambda Agents for Jenkins.
Assuming you already deployed at least one Lambda agent
Jenkins will need to be allowed to invoke Lambda Functions. So you must :
- configure AWS Credentials in Jenkins using the CloudBees AWS Credentials plugin
- unless your Jenkins is running as an AWS Resource (ECS task, EKS pod, EC2 instance)
Those credentials will match an IAM User or a IAM Role which will need to have the following permissions :
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- lambda:ListFunctions
Resource:
- "*"
- Effect: Allow
Action:
- lambda:InvokeFunction
Resource:
- "*"
See below in the "Advanced Configuration" part for finer-grained IAM permissions.
- Install this plugin
- Go the Cloud configuration
- on older Jenkins versions, go at the bottom of
/configure
page - on more recent Jenkins versions, go to the
/configureClouds
page
- on older Jenkins versions, go at the bottom of
- Add a Cloud of type AWS Lambda Cloud
- Configure the cloud
- use AWS Credentials or let is empty if your Jenkins is running in an AWS Context
- configure the region
- optional: configure the Jenkins URL (will default to Jenkins URL) in cases where your need internal communications. The Lambda functions must be able to reach thsi URL.
- Agent connection timeout : The time in seconds to wait before giving up on an agent connection (typically idle agent).
- Add a Lambda Function
- Add as much functions as you need (type, tools, size, ...)
If you need to declare functions in other contexts (credentials, regions, ...), you will have to create another Cloud.
using basic groovy script
Basic configuration using default credentials and default region if Jenkins is running as an ECS task or an EC2 instance with 2 samples lambdas :
jnlp-agent-git-bash
with labellambda-git
jnlp-agent-git-bash-node
with labellambda-node
import io.jenkins.plugins.aws.lambda.cloud.LambdaCloud;
import io.jenkins.plugins.aws.lambda.cloud.LambdaFunction;
import jenkins.model.Jenkins
jenkins = jenkins.model.Jenkins.get()
c = new LambdaCloud("aws-lambdas", null, '')
f = new LambdaFunction('jnlp-agent-git-bash', "lambda-git");
f2 = new LambdaFunction('jnlp-agent-git-bash-node', "lambda-node");
c.setFunctions([f, f2]);
jenkins.clouds.add(c);
jenkins.save()
It is a good practice to restrict the permissions on your Jenkins and allow it to execute only the functions that you have planned it to use. A nice way to do it would be tag your Lambda functions and apply IAM restrictions by conditions on those tags. Unfortunately, AWS Lambda does not support Authorization based on tags https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_aws-services-that-work-with-iam.html. So to apply fine grain permissions, we will need to use the names of the functions.
In the deployment sample (serverless), all lambdas are deployed with jnlp-agent
prefix.
So to limit the invocation permissions of Jenkins to only those Lambda functions, we can use this policy :
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- lambda:ListFunctions
Resource:
- "*"
- Effect: Allow
Action:
- lambda:InvokeFunction
Resource:
- !Sub "arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:jnlp-agent-*"
this is the Cloudformation compatible format of an IAM Policy. Replace !Sub
, ${AWS::Region}
and ${AWS::AccountId}
if needed.
By default Jenkins do estimate load to avoid over-provisioning of cloud nodes. This plugin will use its own provisioning strategy by default, with this strategy, a new node is created on Lambda as soon as NodeProvisioner detects need for more agents. In worse scenarios, this will results in some extra nodes provisioned on Lambda, which will be shortly terminated.
If you want to turn off this Strategy you can set SystemProperty io.jenkins.plugins.aws.lambda.cloud.lambdaCloudProvisionerStrategy.disable=true
- check that you have defined the label
xxxxx
in one of your Lambda Function. You can provide a list of labels for the same function separated by whitespaces (not commas). - Check the Jenkins logs or Lambda Agent Logs (see below)
Look above at the important notice in "Jenkins Required Configuration".
Depending on your system, you can consult logs :
- on the
/log/all
url - or configure a recorder on the
io.jenkins.plugins.aws.lambda.cloud
package with at least INFO level
If you see errors like is not authorized to perform: lambda:InvokeFunction on resource:
, check the permissions of the credentials you set on the cloud (explicitly or implicitly).
Look at the Cloudwatch console.
TO COME : a build wrapper to give you a quick access to the lambda log group/stream of execution.