Skip to content

Use static IP for daemon tasks. - #4881

Merged
aviral92 merged 2 commits into
aws:devfrom
aviral92:daemonIP2
Feb 26, 2026
Merged

aviral92 merged 2 commits into
aws:devfrom
aviral92:daemonIP2

Conversation

@aviral92

@aviral92 aviral92 commented Feb 24, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Related bridge plugin PR(this needs to be merged first) - aws/amazon-ecs-cni-plugins#120
ECS agent now tells the IPAM plugin assign 169.254.172.2 to this daemon container instead of pick any available IP from the subnet. ALso, Implements IMDS (Instance Metadata Service) blocking for daemon-bridge network mode tasks by integrating the functionality into the bridge CNI plugin, following the same pattern as the vpc-branch-eni plugin.

Implementation details

Overview

This PR enables daemon tasks to use a static IP address (169.254.172.2) and adds support for connected subnet routes in awsvpc tasks to enable direct communication between replica and daemon tasks via the bridge network.

Key Changes

1. Static IP Assignment for Daemon Tasks

  • Daemon tasks now receive a fixed IP: 169.254.172.2/32 (IPv4) and fd00:ec2::172:2/128 (IPv6)
  • Ensures predictable addressing for daemon-to-replica communication
  • Updated createDaemonBridgePluginConfig() to set IPV4Address and IPV6Address in IPAM config

2. Connected Subnet Route Support

  • Added ConnectedSubnetMaskSizeIPv4 and ConnectedSubnetMaskSizeIPv6 fields to IPAMConfig
  • Enables CNI plugin to create connected subnet routes (169.254.172.0/22) for direct bridge communication
  • Routes are conditionally added based on daemon namespace presence

3. Daemon Namespace Detection

  • Converted createBridgePluginConfig() to a method on common struct for access to nsUtil
  • Added daemonNamespaceExists() method using nsUtil.NSExists() for robust namespace checking
  • Awsvpc tasks check for daemon namespace and configure routes accordingly:
    • Daemon exists: maskSize = 22 (IPv4), 112 (IPv6) → enables connected subnet routes
    • No daemon: maskSize = 0 → skips connected subnet routes (awsvpc-only mode)

4. Block IMDS on Daemon-bridge tasks similar to awsvpc tasks which already block IMDS via the vpc-branch-eni plugin.

  • ECS Agent Changes:

    • Added BlockIMDS field to BridgeConfig struct
    • Set BlockIMDS: true in createDaemonBridgePluginConfig() for daemon-bridge tasks
    • Added logging to track when IMDS blocking is enabled
  • Bridge CNI Plugin Changes:

    • Added BlockInstanceMetadata() method to Engine interface
    • Blocks both IPv4 (169.254.169.254/32) and IPv6 (fd00:ec2::254/128) IMDS endpoints using blackhole routes
    • Follows vpc-branch-eni plugin pattern for consistency
    • Added detailed logging for troubleshooting

Testing

  • Unit tests updated to reflect new static IP assignments
  • Validated on live instances with daemon+awsvpc and awsvpc-only configurations
  • Updated existing unit tests to verify BlockIMDS: true is set correctly
  • Added new unit test TestBlockInstanceMetadata for bridge plugin
  • Validated on live instances:
    • Host with daemon + awsvpc tasks: Both have IMDS blocked ✅
    • Host with awsvpc-only tasks: No impact, IMDS blocked via vpc-branch-eni ✅

Testing

UTs and build an AMI with the ecs-bridge changes for E2E testing.

New tests cover the changes: Yes.

Description for the changelog

Use static IP for daemon tasks.

Additional Information

Does this PR include breaking model changes? If so, Have you added transformation functions?
No.

Does this PR include the addition of new environment variables in the README?
No.

Licensing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Apendix :

How the IPAM Plugin Works

The IPAM (IP Address Management) plugin has two modes:

  1. Static IP Mode (NEW - what this commit enables):
    if assignedAddress.IP != nil {
    // IP was specified in configuration, try to assign this ip as used
    err := ipManager.Assign(assignedAddress.IP.String(), conf.ID)
    }

    • When IPV4Address is set in the config, IPAM tries to reserve that specific IP
    • If already in use, it returns an error
    • Otherwise, it marks that IP as assigned in the database (eni-ipam.db)
  2. Dynamic IP Mode (OLD - before this commit):
    else {
    // Get the next ip from db based on the last used ip
    nextIP, err := getIPV4AddressFromDB(ipManager, conf)
    assignedAddress.IP = net.ParseIP(nextIP)
    }

    • When IPV4Address is NOT set, IPAM allocates the next available IP from the subnet

Why Environment Variables Were Added

The environment variables tell the IPAM plugin where to store its state:

m.common.os.Setenv(IPAMDataPathEnv, filepath.Join(m.common.stateDBDir, IPAMDataFileName))
// Sets: IPAM_DB_PATH=/var/lib/ecs/data/eni-ipam.db

Purpose:

  • The IPAM plugin maintains a database (eni-ipam.db) to track which IPs are assigned
  • This prevents IP conflicts when multiple containers are created
  • For static IPs, the database ensures the same IP (169.254.172.2) is consistently assigned to daemon tasks
  • The database persists across agent restarts

@JoseVillalta JoseVillalta left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a question.

JoseVillalta
JoseVillalta previously approved these changes Feb 24, 2026
aaithal
aaithal previously approved these changes Feb 24, 2026
- Always block IMDS access for daemon-bridge tasks
- Follows same pattern as awsvpc on managed instances
- Uses blackhole route (netlink RouteAdd with RTN_BLACKHOLE)
- Blocks access to both IPv4 (169.254.169.254) and IPv6 (fd00:ec2::254) IMDS endpoints
- Executes inside daemon namespace using ExecInNSPath
- Add comprehensive unit tests for IMDS blocking functionality

@JoseVillalta JoseVillalta left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@aviral92
aviral92 enabled auto-merge February 26, 2026 07:05
@aviral92
aviral92 merged commit e22e987 into aws:dev Feb 26, 2026
44 checks passed
@prateekchaudhry prateekchaudhry mentioned this pull request Mar 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants