Merge pull request #212 from Daylily-Informatics/refactor/use-iam-groups

Refactor/use iam groups

Author: iamh2o

README.md

@@ -102,16 +102,29 @@ From the `Iam -> Users` console, create a new user.
 - Review the confiirmation page, and click `Create user`.
 - On the next page, capture the `Console sign-in URL`, `username`, and `password`. You will need these to log in as the `daylily-service` user.
 
-### Attach Permissiong & Policies To The `daylily-service` User
+### Attach Permissions & Policies via an IAM Group (recommended)
 _still as the admin user_
 
+Daylily now prefers using an IAM *group* (default: `daylily-ephemeral-cluster`) and
+making the `daylily-service` user (and any other operators) members of that group.
+
 #### Permissions
 
-- Navigate to the `IAM -> Users` console, click on the `daylily-service` user.
-- Click on the `Add permissions` button, then select `Add permission`, `Attach policies directly`.
-- Search for `AmazonQDeveloperAccess` , select and add.
-- Search for `AmazonEC2SpotFleetAutoscaleRole`, select and add.
-- Search for `AmazonEC2SpotFleetTaggingRole`, select and add.
+- Navigate to `IAM -> User groups` and create a group named `daylily-ephemeral-cluster`.
+- Attach the following AWS managed policies to the **group**:
+  - `AmazonQDeveloperAccess`
+  - `AmazonEC2SpotFleetAutoscaleRole`
+  - `AmazonEC2SpotFleetTaggingRole`
+- Add the `daylily-service` user to the group.
+
+> Legacy note: attaching policies directly to the user still works, but is discouraged.
+
+Migration (recommended):
+- Ensure `daylily-ephemeral-cluster` group exists and has the required policies attached.
+- Add `daylily-service` (and any other operators) to the group.
+- Optionally remove old direct user attachments once verified:
+  - Managed policies: `aws iam list-attached-user-policies --user-name daylily-service` then `aws iam detach-user-policy ...`
+  - Inline policies: `aws iam list-user-policies --user-name daylily-service` then `aws iam delete-user-policy ...`
 
 #### Create Service Linked Role `VERY IMPORTANT`
 
@@ -131,8 +144,8 @@ aws iam create-service-linked-role --aws-service-name spot.amazonaws.com
 
 #### Inline Policy
 __**note:**__ [please consult the parallel cluster docs for fine grained permissions control, the below is a broad approach](https://docs.aws.amazon.com/parallelcluster/latest/ug/iam-roles-in-parallelcluster-v3.html).
-- Navigate to the `IAM -> Users` console, click on the `daylily-service` user.
-- Click on the `Add permissions` button, then select `create inline policy`.
+- Navigate to the `IAM -> User groups` console, click on the `daylily-ephemeral-cluster` group.
+- Click on `Add permissions` and select `Create inline policy`.
 - Click on the `JSON` bubble button.
 - Delete the auto-populated json in the editor window, and paste this json into the editor (replace 3 instances of  <AWS_ACCOUNT_ID> with your new account number, an integer found in the upper right dropdown).
 
@@ -141,6 +154,10 @@ __**note:**__ [please consult the parallel cluster docs for fine grained permiss
 - `click next`
 - Name the policy `daylily-service-cluster-policy` (not formally mandatory, but advised to bypass various warnings in future steps), then click `Create policy`.
 
+Alternative (preferred): use the provided admin scripts to create/attach the Daylily policies to your group:
+- `bin/admin/daylily_ephemeral_cluster_bootstrap_global.sh --user daylily-service --group daylily-ephemeral-cluster`
+- `bin/admin/daylily_ephemeral_cluster_bootstrap_region.sh --region <REGION> --user daylily-service --group daylily-ephemeral-cluster`
+
 
 ### Additional AWS Considerations (also will need _admin_ intervention)
 #### Quotas
@@ -1353,15 +1370,24 @@ All tools involved in `daylily-ephemeral-cluster` can be managed in such a way t
  ### One Time, Create Policy
  ```bash
 export AWS_PROFILE=YOURADMINUSERPROFILE
-aws iam create-policy \\n  --policy-name DaylilyCostRead \\n  --policy-document file://config/aws/generate_cluster_report.json
-export AWS_PROFILE=<daylily-service>
+aws iam create-policy \
+  --policy-name DaylilyCostRead \
+  --policy-document file://config/aws/generate_cluster_report.json
  ```
 
- ### Per User (ie: daylily-service)
+### Per Group / User membership (recommended)
  ```bash
 export AWS_PROFILE=YOURADMINUSERPROFILE
-aws iam attach-user-policy \\n  --user-name daylily-service \\n  --policy-arn arn:aws:iam::108782052779:policy/DaylilyCostRead
-export AWS_PROFILE=<daylily-service>
+
+GROUP_NAME=daylily-ephemeral-cluster
+USER_NAME=daylily-service
+ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
+
+aws iam create-group --group-name "$GROUP_NAME" >/dev/null 2>&1 || true
+aws iam attach-group-policy \
+  --group-name "$GROUP_NAME" \
+  --policy-arn "arn:aws:iam::${ACCOUNT_ID}:policy/DaylilyCostRead"
+aws iam add-user-to-group --user-name "$USER_NAME" --group-name "$GROUP_NAME"
  ```
  
  
@@ -1370,4 +1396,4 @@ export AWS_PROFILE=<daylily-service>
 
 _named in honor of Margaret Oakley Dahoff_ 
  
- 
+ X

bin/admin/daylily_ephemeral_cluster_bootstrap_global.sh

@@ -6,24 +6,31 @@ usage() {
 Bootstrap global IAM policy for Daylily ephemeral cluster workflows.
 
 This script creates or updates the global Daylily managed policy and optionally
-attaches it to a specified IAM user. Run this once per AWS account, or again if
+attaches it to a specified IAM *group* (recommended) and ensures the target IAM
+user is a member of that group. Run this once per AWS account, or again if
 updates to the policy are released.
 
 USAGE:
-  daylily_ephemeral_cluster_bootstrap_global.sh --user USERNAME [--profile PROFILE]
+  daylily_ephemeral_cluster_bootstrap_global.sh \
+    --user USERNAME \
+    [--group GROUP] \
+    [--profile PROFILE]
 
 OPTIONS:
-  --user      IAM username to attach the Daylily global policy to (required)
+  --user      IAM username to grant access to (required)
+  --group     IAM group to attach the Daylily global policy to (default: daylily-ephemeral-cluster)
   --profile   AWS CLI profile with admin rights (optional)
 USAGE
 }
 
 USER_NAME=""
+GROUP_NAME="daylily-ephemeral-cluster"
 PROFILE="${AWS_PROFILE:-}"
 
 while (( $# )); do
   case "$1" in
     --user) USER_NAME="${2:-}"; shift 2 ;;
+	--group) GROUP_NAME="${2:-}"; shift 2 ;;
     --profile) PROFILE="${2:-}"; shift 2 ;;
     -h|--help) usage; exit 0 ;;
     *) echo "Unknown arg: $1" >&2; usage >&2; exit 2 ;;
@@ -32,6 +39,8 @@ done
 
 : "${USER_NAME:?ERR: --user required}"
 
+[[ -n "${GROUP_NAME}" ]] || { echo "ERR: --group cannot be empty" >&2; exit 2; }
+
 AWS=(aws)
 [[ -n "$PROFILE" ]] && AWS+=(--profile "$PROFILE")
 
@@ -117,10 +126,59 @@ create_or_update_policy() {
 
 GLOBAL_ARN="$(create_or_update_policy "${GLOBAL_POLICY_NAME}" "${GLOBAL_POLICY_DOC}")"
 
->&2 echo "Attaching ${GLOBAL_POLICY_NAME} to user ${USER_NAME}"
-"${AWS[@]}" iam attach-user-policy --user-name "${USER_NAME}" --policy-arn "${GLOBAL_ARN}" || true
+ensure_user_exists() {
+  local user="$1"
+  "${AWS[@]}" iam get-user --user-name "${user}" >/dev/null 2>&1 || {
+    echo "ERR: IAM user '${user}' not found." >&2
+    exit 4
+  }
+}
+
+ensure_group_exists() {
+  local group="$1"
+  if ! "${AWS[@]}" iam get-group --group-name "${group}" >/dev/null 2>&1; then
+    >&2 echo "Creating IAM group: ${group}"
+    "${AWS[@]}" iam create-group --group-name "${group}" >/dev/null
+  fi
+}
+
+ensure_policy_attached_to_group() {
+  local group="$1" policy_arn="$2"
+  local attached
+  attached=$(
+    "${AWS[@]}" iam list-attached-group-policies --group-name "${group}" \
+      --query "AttachedPolicies[?PolicyArn=='${policy_arn}'] | length(@)" --output text 2>/dev/null || echo "0"
+  )
+  if [[ "${attached}" == "0" ]]; then
+    >&2 echo "Attaching ${GLOBAL_POLICY_NAME} to group ${group}"
+    "${AWS[@]}" iam attach-group-policy --group-name "${group}" --policy-arn "${policy_arn}"
+  else
+    >&2 echo "${GLOBAL_POLICY_NAME} already attached to group ${group}"
+  fi
+}
+
+ensure_user_in_group() {
+  local user="$1" group="$2"
+  local in_group
+  in_group=$(
+    "${AWS[@]}" iam list-groups-for-user --user-name "${user}" \
+      --query "Groups[?GroupName=='${group}'] | length(@)" --output text 2>/dev/null || echo "0"
+  )
+  if [[ "${in_group}" == "0" ]]; then
+    >&2 echo "Adding user ${user} to group ${group}"
+    "${AWS[@]}" iam add-user-to-group --user-name "${user}" --group-name "${group}"
+  else
+    >&2 echo "User ${user} already in group ${group}"
+  fi
+}
+
+ensure_user_exists "${USER_NAME}"
+ensure_group_exists "${GROUP_NAME}"
+ensure_policy_attached_to_group "${GROUP_NAME}" "${GLOBAL_ARN}"
+ensure_user_in_group "${USER_NAME}" "${GROUP_NAME}"
 
 cat <<SUMMARY
 ✅ Done.
-  - ${GLOBAL_POLICY_NAME}: ${GLOBAL_ARN} (attached to ${USER_NAME})
+  - ${GLOBAL_POLICY_NAME}: ${GLOBAL_ARN} (attached to IAM group ${GROUP_NAME})
+  - IAM user ${USER_NAME} is a member of ${GROUP_NAME}
 SUMMARY

bin/admin/daylily_ephemeral_cluster_bootstrap_region.sh

@@ -6,28 +6,34 @@ usage() {
 Bootstrap region-scoped IAM policies for Daylily ephemeral cluster workflows.
 
 This script creates or updates the Daylily region policy alongside the
-ParallelCluster Lambda adjust policy, and attaches the region policy to a given
-user. Run this once per AWS region in which you operate Daylily clusters.
+ParallelCluster Lambda adjust policy.
+
+It attaches the region policy to an IAM *group* (recommended) and ensures the
+target IAM user is a member of that group. Run this once per AWS region in which
+you operate Daylily clusters.
 
 USAGE:
   daylily_ephemeral_cluster_bootstrap_region.sh \\
-    --region REGION --user USERNAME [--profile PROFILE]
+    --region REGION --user USERNAME [--group GROUP] [--profile PROFILE]
 
 OPTIONS:
   --region    AWS region to scope the policy to (required)
-  --user      IAM username to attach the Daylily region policy to (required)
+  --user      IAM username to grant access to (required)
+  --group     IAM group to attach the Daylily region policy to (default: daylily-ephemeral-cluster)
   --profile   AWS CLI profile with admin rights (optional)
 USAGE
 }
 
 USER_NAME=""
 REGION=""
+GROUP_NAME="daylily-ephemeral-cluster"
 PROFILE="${AWS_PROFILE:-}"
 
 while (( $# )); do
   case "$1" in
     --user) USER_NAME="${2:-}"; shift 2 ;;
     --region) REGION="${2:-}"; shift 2 ;;
+	--group) GROUP_NAME="${2:-}"; shift 2 ;;
     --profile) PROFILE="${2:-}"; shift 2 ;;
     -h|--help) usage; exit 0 ;;
     *) echo "Unknown arg: $1" >&2; usage >&2; exit 2 ;;
@@ -36,6 +42,7 @@ done
 
 : "${USER_NAME:?ERR: --user required}"
 : "${REGION:?ERR: --region required}"
+[[ -n "${GROUP_NAME}" ]] || { echo "ERR: --group cannot be empty" >&2; exit 2; }
 
 AWS=(aws)
 [[ -n "$PROFILE" ]] && AWS+=(--profile "$PROFILE")
@@ -158,11 +165,60 @@ else
   done
 fi
 
-echo "Attaching ${REGION_POLICY_NAME} to user ${USER_NAME}"
-"${AWS[@]}" iam attach-user-policy --user-name "${USER_NAME}" --policy-arn "${REGION_ARN}" || true
+ensure_user_exists() {
+  local user="$1"
+  "${AWS[@]}" iam get-user --user-name "${user}" >/dev/null 2>&1 || {
+    echo "ERR: IAM user '${user}' not found." >&2
+    exit 4
+  }
+}
+
+ensure_group_exists() {
+  local group="$1"
+  if ! "${AWS[@]}" iam get-group --group-name "${group}" >/dev/null 2>&1; then
+    echo "Creating IAM group: ${group}"
+    "${AWS[@]}" iam create-group --group-name "${group}" >/dev/null
+  fi
+}
+
+ensure_policy_attached_to_group() {
+  local group="$1" policy_arn="$2"
+  local attached
+  attached=$(
+    "${AWS[@]}" iam list-attached-group-policies --group-name "${group}" \
+      --query "AttachedPolicies[?PolicyArn=='${policy_arn}'] | length(@)" --output text 2>/dev/null || echo "0"
+  )
+  if [[ "${attached}" == "0" ]]; then
+    echo "Attaching ${REGION_POLICY_NAME} to group ${group}"
+    "${AWS[@]}" iam attach-group-policy --group-name "${group}" --policy-arn "${policy_arn}"
+  else
+    echo "${REGION_POLICY_NAME} already attached to group ${group}"
+  fi
+}
+
+ensure_user_in_group() {
+  local user="$1" group="$2"
+  local in_group
+  in_group=$(
+    "${AWS[@]}" iam list-groups-for-user --user-name "${user}" \
+      --query "Groups[?GroupName=='${group}'] | length(@)" --output text 2>/dev/null || echo "0"
+  )
+  if [[ "${in_group}" == "0" ]]; then
+    echo "Adding user ${user} to group ${group}"
+    "${AWS[@]}" iam add-user-to-group --user-name "${user}" --group-name "${group}"
+  else
+    echo "User ${user} already in group ${group}"
+  fi
+}
+
+ensure_user_exists "${USER_NAME}"
+ensure_group_exists "${GROUP_NAME}"
+ensure_policy_attached_to_group "${GROUP_NAME}" "${REGION_ARN}"
+ensure_user_in_group "${USER_NAME}" "${GROUP_NAME}"
 
 cat <<SUMMARY
 ✅ Done.
-  - ${REGION_POLICY_NAME}: ${REGION_ARN} (attached to ${USER_NAME})
+  - ${REGION_POLICY_NAME}: ${REGION_ARN} (attached to IAM group ${GROUP_NAME})
+  - IAM user ${USER_NAME} is a member of ${GROUP_NAME}
   - ${ADJUST_POLICY_NAME}: ${ADJUST_ARN}
 SUMMARY