[danehlim]

Summary

Add to the ecs-agent module a new type FailureTracker, which is used to:

• track whether an operation has been in a failure state continuously for longer than a specified time threshold (i.e., 1 minute), and
• if so, emit a metric at a specified cadence (i.e., every 1 minute)

Use this new type to monitor failures in:

• resolving ECS endpoints
• establishing connection with ACS
• establishing connection with TCS

Implementation details

• Define FailureTracker and its functions
• For each FailureTracker, record success upon each successful operation and record failure upon each terminal failure mode
• Use functional options pattern to supply FailureTracker to structs which manage calling DiscoverPollEndpoint ECS API, connecting to ACS, and connecting to TCS (making supplying FailureTracker optional by design)

Out of scope

The following is out of scope of this pull request:

• Updating ECS Agent (i.e., agent module) to supply FailureTracker when initializing its ACS session and TCS session and kick off metric emission logic
• Adding support for other functional options to ACS session and TCS session besides newly added WithDPEFailureTracker, WithACSConnFailureTracker, and WithTCSConnFailureTracker
• Removing any metrics already defined in ecs-agent/metrics/constants.go which may contain overlap with the newly added DPEConnectivityFailure, ACSConnectivityFailure, and TCSConnectivityFailure

Testing

New tests cover the changes: yes

Description for the changelog

N/A

Additional Information

Does this PR include breaking model changes? If so, Have you added transformation functions?

No

Does this PR include the addition of new environment variables in the README?

No

Licensing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.