Skip to content
Snippets Groups Projects

migrating kagra members to ldap.igwn.org

Merged Alexander Pace requested to merge migrated-kagra-ldap into master
1 unresolved thread

This modifies the ldap-querying script that controls user group memberships to get KAGRA users from ldap.igwn.org and off gw-astronomy as outlined here: https://wiki.ligo.org/AuthProject/AccessingKAGRAonIGWNLDAP

Merge request reports

Pipeline #691645 passed

Pipeline passed for 9f74b3e3 on migrated-kagra-ldap

Test coverage 72.00% (0.00%) from 1 job

Merged by Alexander PaceAlexander Pace 2 weeks ago (Dec 24, 2024 4:43pm UTC)

Loading

Pipeline #691646 passed

Pipeline passed for 0035683b on master

Test coverage 72.00% (0.00%) from 1 job

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Here's the output when I ran it for the first time:

    Created genericldapuser for tomotada.akutsu@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for yuta.michimura@shibbi.pki.itc.u-tokyo.ac.jp
    User yuta.michimura@shibbi.pki.itc.u-tokyo.ac.jp updated
    Created genericldapuser for yousuke.itoh@shibbi.pki.itc.u-tokyo.ac.jp
    Assigning cert with subject /dc=org/dc=cilogon/o=kagra/cn=yousuke itoh a25345671 to ldapuser KAGRA- yousuke.itoh@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for masahiro.kamiizumi@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for ryutaro.takahashi@shibbi.pki.itc.u-tokyo.ac.jp
    Assigning cert with subject /dc=org/dc=cilogon/o=kagra operated by the university of tokyo/cn=ryutaro takahashi t315418 to ldapuser KAGRA- ryutaro.takahashi@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for hiroyuki.nakano@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for ken-ichi.oohara@shibbi.pki.itc.u-tokyo.ac.jp
    User ken-ichi.oohara@shibbi.pki.itc.u-tokyo.ac.jp updated
    Assigning cert with subject /dc=org/dc=cilogon/o=kagra/cn=ken-ichi oohara a19191816 to ldapuser KAGRA- ken-ichi.oohara@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for hirotaka.yuzurihara@shibbi.pki.itc.u-tokyo.ac.jp
    Assigning cert with subject /dc=org/dc=cilogon/o=kagra/cn=hirotaka yuzurihara b21417647 to ldapuser KAGRA- hirotaka.yuzurihara@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for hirotaka.takahashi@shibbi.pki.itc.u-tokyo.ac.jp
    Assigning cert with subject /dc=org/dc=cilogon/o=kagra/cn=hirotaka takahashi a17364371 to ldapuser KAGRA- hirotaka.takahashi@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for masaki.ando@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for yoshio.saito@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for koh.ueno@shibbi.pki.itc.u-tokyo.ac.jp
    Assigning cert with subject /dc=org/dc=cilogon/o=kagra/cn=koh ueno e182498 to ldapuser KAGRA- koh.ueno@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for tatsuya.narikawa@shibbi.pki.itc.u-tokyo.ac.jp
    Assigning cert with subject /dc=org/dc=cilogon/o=kagra/cn=tatsuya narikawa a51480991 to ldapuser KAGRA- tatsuya.narikawa@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for nami.uchikata@shibbi.pki.itc.u-tokyo.ac.jp
    Assigning cert with subject /dc=org/dc=cilogon/o=kagra/cn=nami uchikata a45167756 to ldapuser KAGRA- nami.uchikata@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for hisaaki.shinkai@shibbi.pki.itc.u-tokyo.ac.jp
    Assigning cert with subject /dc=org/dc=cilogon/o=kagra/cn=hisaaki shinkai b23691217 to ldapuser KAGRA- hisaaki.shinkai@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for yu-kuang.chu@shibbi.pki.itc.u-tokyo.ac.jp
    Assigning cert with subject /dc=org/dc=cilogon/o=kagra operated by the university of tokyo/cn=yu-kuang chu t315898 to ldapuser KAGRA- yu-kuang.chu@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for takahiro.sawada@shibbi.pki.itc.u-tokyo.ac.jp
    Assigning cert with subject /dc=org/dc=cilogon/o=kagra/cn=takahiro sawada a21480346 to ldapuser KAGRA- takahiro.sawada@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for kipp.cannon@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for tomoyuki.uehara@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for shoichi.oshino@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for june-gyu.park@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for chia-jui.chou@shibbi.pki.itc.u-tokyo.ac.jp
    Assigning cert with subject /dc=org/dc=cilogon/o=kagra/cn=chia-jui chou b48782412 to ldapuser KAGRA- chia-jui.chou@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for chang-hee.kim@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for kyujin.kwak@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for lijing.shao@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for yuichiro.sekiguchi@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for yi.yang@shibbi.pki.itc.u-tokyo.ac.jp
    User yi.yang@shibbi.pki.itc.u-tokyo.ac.jp updated
    Assigning cert with subject /dc=org/dc=cilogon/o=kagra/cn=yi yang b45162672 to ldapuser KAGRA- yi.yang@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for chun-che.lin@shibbi.pki.itc.u-tokyo.ac.jp
    User chun-che.lin@shibbi.pki.itc.u-tokyo.ac.jp updated
    Assigning cert with subject /dc=org/dc=cilogon/o=kagra/cn=chun-che lin b47908452 to ldapuser KAGRA- chun-che.lin@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for luca.baiotti@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for dan.chen@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for kenta.tanaka@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for john-j.oh@shibbi.pki.itc.u-tokyo.ac.jp
    User john-j.oh@shibbi.pki.itc.u-tokyo.ac.jp updated
    Created genericldapuser for yu-chiung.lin@shibbi.pki.itc.u-tokyo.ac.jp
    Assigning cert with subject /dc=org/dc=cilogon/o=kagra/cn=yu-chiung lin e9363 to ldapuser KAGRA- yu-chiung.lin@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for shingo.fujii@shibbi.pki.itc.u-tokyo.ac.jp
    User shingo.fujii@shibbi.pki.itc.u-tokyo.ac.jp updated
    Created genericldapuser for marco.meyer@shibbi.pki.itc.u-tokyo.ac.jp
    User marco.meyer@shibbi.pki.itc.u-tokyo.ac.jp updated
    Assigning cert with subject /dc=org/dc=cilogon/o=kagra/cn=marco meyer e10075 to ldapuser KAGRA- marco.meyer@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for sangin.kim@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for michael.manske@shibbi.pki.itc.u-tokyo.ac.jp
    Assigning cert with subject /dc=org/dc=cilogon/o=kagra/cn=michael manske e87347 to ldapuser KAGRA- michael.manske@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for cheng-min.chen@shibbi.pki.itc.u-tokyo.ac.jp
    Assigning cert with subject /dc=org/dc=cilogon/o=kagra/cn=cheng-min chen e137283 to ldapuser KAGRA- cheng-min.chen@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for nim-ki.wong@shibbi.pki.itc.u-tokyo.ac.jp
    User nim-ki.wong@shibbi.pki.itc.u-tokyo.ac.jp updated
    Assigning cert with subject /dc=org/dc=cilogon/o=kagra/cn=nim-ki wong e136696 to ldapuser KAGRA- nim-ki.wong@shibbi.pki.itc.u-tokyo.ac.jp
    Created genericldapuser for gyoik.kim@shibbi.pki.itc.u-tokyo.ac.jp
    User gyoik.kim@shibbi.pki.itc.u-tokyo.ac.jp updated
    Created genericldapuser for heather.fong@ligo.org

    It created a new genericldapuser object for each member in the new ldap, which is to be expected since it's tied to the user entry's ldap_dn.

    I'm going to use Kipp as an example here since he has an entry in the old and new ldaps:

    In : from django.contrib.auth.models import User, Group, Permission
    
    In : kipp = User.objects.filter(username__contains='kipp').last()
    
    In : kipp
    Out: <User: kipp.cannon@shibbi.pki.itc.u-tokyo.ac.jp>

    He does have two genericldapuser objects associated with his kagra account, one for gw-astronomy, and one for ldap.igwn.org:

    In : for g in kipp.genericldapuser_set.all():
    ...:     print(g.__dict__)
    ...:
    {'_state': <django.db.models.base.ModelState object at 0x7f89698e0090>, 'id': 5888, 'ldap_dn': 'employeeNumber=KL10239,ou=people,o=KAGRA-LIGO,o=CO,dc=gwastronomy-data,dc=cgca,dc=uwm,dc=edu', 'ldap_member_id': 2, 'user_id': 5898}
    {'_state': <django.db.models.base.ModelState object at 0x7f89698471d0>, 'id': 7353, 'ldap_dn': 'voPersonID=KL10239,ou=people,o=KAGRA,dc=igwn,dc=org', 'ldap_member_id': 2, 'user_id': 5898}

    A user's access to gracedb (basically if they're active) is tied to them being part of the internal_users group. Kipp currently is:

    In : kipp.groups.all()
    Out: <QuerySet [<Group: internal_users>, <Group: rrt_members>, <Group: priority_alerts>]>

    To test that the script is working, I'm going to manually remove him from the internal_users group, and then run the script again to see if he gets added back:

    In : iu = Group.objects.get(name='internal_users')
    
    In : iu
    Out: <Group: internal_users>
    
    In : kipp.groups.remove(iu)
    
    In : kipp.groups.all()
    Out: <QuerySet [<Group: rrt_members>, <Group: priority_alerts>]>

    Rerunning the script again produces only this output:

    ❯ python3 manage.py update_user_accounts_from_ligo_ldap kagra
    Refreshing users from kagra LDAP at 2024-12-04 20:25:40.891003
    Adding kipp.cannon@shibbi.pki.itc.u-tokyo.ac.jp to internal_users and rrt_members and priority_alerts

    And then confirming in the console that it works:

    In : from django.contrib.auth.models import User, Group, Permission
    
    In : kipp = User.objects.filter(username__contains='kipp').last()
    
    In : kipp.groups.all()
    Out: <QuerySet [<Group: internal_users>, <Group: rrt_members>, <Group: priority_alerts>]>

    🎆 so it works, as far as I can tell.

    • Two questions for @warren-anderson:

      1. When's the soonest that this should be put into production? The wiki says "All KAGRA members who still need access to LVK services in ldaps://ldap.igwn.org by Dec. 18, 2024."; is December 18th (two weeks from now) a "not earlier than" date for switching GraceDB access for KAGRA users to ldap.igwn.org?

      2. There's the question about what to do with the user accounts that are associated with the gw-astronomy ldap. Meaning, if they have internal_users membership before the switch, and they're not in ldap.igwn.org yet, then their accounts will never be modified. One thing I could do is query for every member of KAGRA in GraceDB, and then manually remove them from the internal_users group (removing their access). And then running this script against ldap.igwn.org will re-add their access, assuming they have a valid membership.

      1. Currently, we are asking all new KAGRA registrants to enroll in both the gw-astronomy and IGWN systems. So gw-astronomy should have everyone, and IGWN should be accumulating everyone. The timeline has three inflection points:
      • Dec. 18 - all KAGRA members are supposed to have enrolled in the new IGWN COmanage instance. Enrollment in gw-astronomy will be turned off. At this point, all those KAGRA members who have enrolled in the IGWN instance will be synced into the IGWN LDAP. At that point, you would be justified in using the IGWN LDAP as the source of truth for KAGRA. However, if there are KAGRA members who enroll in the IGWN instance after this point, and they have a gw-astronomy KAGRA account, they will appear in the IGWN LDAP but will not have their gw-astronomy information (e.g. groups) transported over immediately. If having only people who have all their group memberships migrated correctly is important to you, you might want to not transition yet.
      • Sometime in the first full week of January - the gw-astronomy COmanage instance will cease to exist. Just before it goes offline, we will sync information over from gw-astronomy to IGWN one last time. From this point, any person from KAGRA who enrolls in the IGWN COmanage will NOT have their information moved over. They will have to do the work of rejoining open groups, asking to be re-added to closed groups, re-subscribing to mailing lists, reapplying for LDG accounts, etc. At this point, I can't think of a reason that you would not want to switch over to the IGWN LDAP if you are ready to.
      • Jan. 31, 2025 - The gw-astronomy LDAP will be shut down. If you are relying on it to get information about KAGRA members at that time, you will get no more information. How your application is affected by that, you would know better than I, but my opinion is that all services should be using the IGWN LDAP before this happens.
      1. I'm not sure what the question for me is here for the internal_users group, but hopefully the information above is sufficient for you to figure out your strategy. If not, I'm glad to get on a zoom call or something.
      Edited by Warren Anderson
    • Thanks warren for the clarification. It sounds like I can make the switch after December 18th.

      As for the question about the internal_users group: there was the issue we spoke about on zoom regarding KAGRA members who have not been off-boarded and still had isMemberOf: gw-astronomy:KAGRA-LIGO:members as one of their ldap attributes. What I could do is, once the gw-astronomy ldap is shut down, is just reset the group membership for everyone who is or was in KAGRA, and then sync it again with igwn's ldap to give access to everyone who is actually active. Or, alternatively I just don't do anything and those members from gw-astronomy's ldap who still had access (and are not in the igwn ldap) continue to have access to GraceDB forever. This is more of a policy decision rather than a technical one.

    • Please register or sign in to reply
  • Alexander Pace added 3 commits

    added 3 commits

    Compare with previous version

  • Alexander Pace enabled an automatic merge when all merge checks for 9f74b3e3 pass

    enabled an automatic merge when all merge checks for 9f74b3e3 pass

Please register or sign in to reply
Loading