Setting Up Your First AWS Organization

Alex Freidman
March 29, 2023

The Principle Challenge With Setting Up AWS Organization for the First Time 

As the CTO of a young startup, you may experience a tendency to maintain a rapid pace, making it hard to stop and think about foundations. Among these foundations, the manner in which your AWS Organization and related accounts are structured warrants careful consideration. These areas are easy to implement if you understand the basics, but hard to change once operations are in full swing. 

Why Having A Single AWS Account Is a Problem

1. Future Downtime is Likely

Once you start running with one main AWS account, it’s difficult in the long run to separate the infrastructure into more than one account without causing some downtime or breaking development productivity in the process. Already have clients that use your platform and developers that want to work? Prepare for a painful interruption! 

2. Separating Roles and Access Is Difficult

When you run in a single AWS account, it’s hard to cleanly separate roles and access between pre-prod and production workloads. It’s possible with IAM, but it’s tough to keep the IAM rules updated when the team, along with the IAM rules complexity, grows.

3. Organization-level Policies Are Inaccessible

Organization-level policies, called SCPs (Service Control Policies), are a powerful tool to define guardrails and set limits in a centralized way, for example, you can limit in which AWS regions resources can be deployed in all development accounts or prevent users from deleting Amazon VPC flow logs in all production accounts.

Service Control Policies bring the most value for Organizations with multiple accounts, moreover, SCPs do not apply to the Management Account itself, thus in the case of a single (Management) account, they are not applicable.

Common Obstacles Faced When Setting Up An AWS Organization 

1. There’s No Real Guidance on Where to Begin

When you open your first AWS account it’s hard to understand what’s the best practice, and that instead of having one single all-purpose account on AWS, you should actually be managing multiple accounts. This being said, creating and managing multiple accounts isn’t straightforward in AWS either.

2. Managing Multiple Accounts Used to Be Difficult 

If you’re a long-time AWS user, you might have had a bad experience creating and managing a multi-account AWS Organization, as there was no proper visibility of all the accounts under management, no single sign-on, and no single consolidated billing. However, these days, with the development of AWS Organization and AWS Identity Center, it’s much easier to manage multiple accounts and access them from the get-go—meaning you can reap the benefits from the beginning, without making things overly complicated as a result.

If it’s the first time you’re reading about AWS Identity Center, a small side note - AWS Identity Center, formerly known as AWS Single Sign-On service, helps centrally manage user identities and their access across AWS accounts and applications. For example, it allows you to connect your favorite IDP (Identity Provider) like Active Directory, Google Workspace, and Okta to your AWS Organization. This means that for a little extra leg-work at the beginning, you can create and organize accounts in an organization, consolidate access management across AWS, and potentially manage it with your existing identity provider.

3. You May Feel Like You’re Losing Control

Without proper resource management by the development team, like:

  • having automated processes for creation and destruction of ephemeral dev environments;
  • tagging all resources with a standard agreed set of tags; and
  • restricting uncontrolled access to resource provisioning 

there is a natural fear of losing control of costs or lack of FinOps visibility due to running workloads distributed in multiple accounts.

Similarly, multiple accounts can initially feel harder to manage. This is due to everything suddenly not being in one place and the need to switch between accounts to start searching for that specific data dump you’ve recently created. And that’s only one small example of the confusion that may affect your day-to-day operations. 

4. AWS Documentation Can Be Overwhelming for New Users

At the moment, there are more than 100 services on AWS—each service with its own terminology and separate documentation. Although AWS docs are great, detailed, and very helpful for engineers who already know their way around AWS, for beginners, they are too overwhelming. This is because most of the content focuses on detailed explanations, and users find it hard to locate documentation that outlines best practices. Unfortunately, this is also the case for AWS organizations and, in general, best practices related to your startup’s foundational setup on AWS.

Organization Management Best Practices - How to Build a Multiple Accounts Architecture

Although there are some common hurdles to look out for when setting up an AWS Organization, with the right information, a multi-account foundation can be just as straightforward to set up as a single one. For startups and small to medium RnD teams (with up to a few dozen team members), we propose to start with relatively simple and flat AWS Organization account architecture:

Management Account: Organization-level resources for Organization owners

Common resources for example:

  • AWS Identity Center (AWS SSO)
  • AWS Organization
  • Consolidated Billing

Infrastructure / Security Account: Shared and security resources for privileged users only

Common resources for example:

  • ECR (container registry)
  • CloudTrail logs (audit logs) 
  • Route 53 DNS Zones
  • Dev Account: For non-production workloads
  • Prod Account: For production workloads

We love this simple AWS accounts structure because it features:

  • The separation of Dev and Prod workloads
  • A Management Account that is used solely for the Organization’s management and control
  • A separate account for shared infrastructure, security tools, and assets (even if you don’t have any in place, it’s a good idea to prepare it for the future)
  • A keep-it-simple approach (there are only 4 accounts and OUs)

Benefits of a Multiple Accounts Architecture

Here are the compelling advantages that a multiple accounts architecture like the one described above can bring to your AWS infrastructure:

1. Easy Onboarding

Freelancers can be quickly onboarded to a pre-prod account and granted access to production in the later stages—when trust has been established.

2. Simple Cost Control

With multiple accounts, you immediately know how much your production and each account costs.

3. Better Visibility and Separation of Concerns

When having multiple accounts under one organization, you can test deployment of new components in a separate account without fearing breaking something in the production environment. 

For example, we recently  met a client who struggled to deploy a functioning EKS cluster in a single account containing both Dev and Prod environments. Eventually, we found out that a global setting in AWS System Manager was set for a specific environment which interfered with the EKS worker nodes, making them malfunction.

As a nice bonus - if you’re looking to reduce your security blast radius - with a multiple accounts architecture a breach of a specific account won’t affect the others. 

A Quick Bit on Over Engineering

It’s very possible when setting up your first AWS Organization to over engineer the architecture. 

For example, we met clients recently who wanted to achieve extra efficiency without a clear justification for it. In this specific instance, the clients had chosen to create complex cross-account network architecture—having different teams responsible for different accounts with shared network resources—which eventually slowed down their operational velocity in the cloud. Also, if you think about this architecture for a second, cross-account resources sharing also broke the aforementioned concept of “separation of concerns,” since production and non-production workloads aren’t separate.

Build an Evolvable Structure

It’s a common fear that setting up your first AWS Organization will be hard to implement. However, this can be overcome by choosing a simple design with a minimal number of accounts (see the design suggested above). Our advice would be to use AWS Identity Center for centralized user management, optionally connecting your external IDP (Identity Provider).

Most importantly, don’t be lazy when starting to work with AWS. Setting up a simple Organization with four accounts is relatively easy these days and will save you a lot of time down the road. This being the case, try to avoid overcomplicated architecture, as having too many accounts intertwined with each other too early on may slow you down, as opposed to adding benefits.

Bring in DevOps Experience

We at Opsfleet offer a unique way to help small to mid-size technology startup teams get the foundations for secure and efficient cloud operations. One of the first areas we almost always improve at the beginning of our client engagements is AWS Organization and accounts structure.

Click here to learn more about how Opsfleet can take your startup from zero to a full DevOps team within weeks.



Your DevOps Partners

Scaling a cloud-enabled startup requires DevOps expertise. We partner with your engineering team to help you build and scale your cloud infrastructure.

Contact us
Contact us illustration