TLDR: A product-engineering case study exploring how we improved our enterprise customers’ leverage of Bluesky while simultaneously simplifying the onboarding process through the implementation of Role-Based Access Control (RBAC).
We recently introduced a feature in Bluesky to support RBAC policies and empower our customers with more granular control over what their users can see and do within our platform. In this post, we’ll share the motivation behind the feature, our initial approach, how we realized we needed to pivot from that approach, as well as some challenges we encountered during the deployment phase.
Bluesky’s users have primarily been members of the Data Platform team. These individuals possess deep knowledge, capability, and responsibility to influence what happens within their Snowflake accounts. Platform teams can be small, overseeing numerous Snowflake accounts, each with thousands of warehouses and millions of queries executed daily. Bluesky excels at identifying and prioritizing potential database optimizations, consolidating them into a central location, and simplifying the decision-making process for the Platform team.
During further discussion with our customers, we learned that Platform teams wanted to delegate some of Bluesky’s centralized findings to the individuals directly responsible for implementing them within the Snowflake account. While the Platform team holds ultimate responsibility for the system’s health, the sheer volume of accounts makes tackling all optimizations impractical.
Clearly, our customers wanted to bring more users into Bluesky, but they needed to limit what those users could see in the application. So, what would role-based access control look like if we built it out?
Bluesky has a popular feature that allows our customers to consolidate spending via warehouses and then further consolidate warehouses into ownership groups. These Data Groups became the foundation for our initial attempt at RBAC. We reasoned that since our customers already used groups to consolidate warehouses and delegate based on those groups, why not add storage-level access (database, schema, table) to those groups? Then, all they would need to do is add users to those groups, and there you have it, RBAC! This initially seemed like a sound approach, considering the typical startup mentality of leveraging existing features for new use cases.
The initial enthusiasm waned as we continued discussions with customers about how these groups would function in practice. Here’s what we were looking at:
It soon became obvious that this approach wasn’t feasible. Not only was it incredibly complex for us to implement, considering it wasn’t our core value proposition, but it would also be resource-intensive for our customers to create and manage all of those intricate rules. As with many good ideas, the solution ended up being the simplest approach we could think of.
“What if we just copy their Snowflake account user permissions?”
Nesting, different scopes, existing mappings, no new page to create… This was definitely the way to go (and seems so obvious in hindsight). The only requirement for our customers is to ensure that all of their Snowflake user accounts have email addresses matching the ones they use to log in to Bluesky.
Here is what Snowflake’s role-based access model looks like:
Roles are granted permissions (like own, view, select) on Snowflake objects, and these roles can also be granted to other roles (nesting) and users. To identify all the objects a user has access to and what type of access they have, you have to traverse the entire graph of these relationships.
So, how would we implement this in Bluesky? The first approach we considered was to create a User-Defined Function (UDF). This function would compare the object and the user’s access policies to determine whether to allow or deny access. Initially, we assumed that this approach would be overly complex due to the potential for multiple Snowflake users per Bluesky user and the presence of nested roles, but we quickly realized that was not the case.
However, calculating access on every request would significantly impact data retrieval performance. The results couldn’t be indexed, which would lead to a full table scan of all data within the requested time frame. This essentially rendered this approach a dead end, pushing us towards a solution that maintained latency neutrality. In the end, we opted for the near opposite approach: pre-computing as much as possible, leveraging the fact that storage is generally the most affordable resource these days.
Here’s how the new process unfolded:
When we started testing out this implementation we realized that the user experience for an RBAC-limited user would be quite different from the existing experience where each Bluesky user can see everything in their Snowflake account. After discussions with our customers about how they envisioned the functionality being used (especially in regards to delegation) we decided that the RBAC user needed their own user experience that was focused on just the work they were expected to follow up on.
Our normal users will see quite a bit of information and have access to many tools that just aren't relevant to a limited user. Here’s what they see when they login:
But these new limited users don’t have access to see account-wide information and aren’t decision makers at that scope. They usually have a single team's worth of resources to manage and thus need an experience that focuses their attention on just what they can control. This is what it looks like to a limited user.
Overall this view provides a much more simplified and streamlined experience with only the information they need to stay on top of their resources.
Okay, our design is coded up, and initial testing looks promising! Time to roll this out to our customers… But, unfortunately, not everything went as smoothly as we anticipated. What challenges did we encounter during the deployment phase?
Problem 1: Incomplete Communication: We informed our customers that users need valid email addresses in their Snowflake user details. However, we should have proactively identified and notified them of users who needed valid email addresses. This resulted in individuals within their organizations logging in with service accounts (which often use numerical IDs in the email field) and unable to see any data.
Lesson Learned: We strive to provide exceptional customer service by proactively identifying and communicating potential roadblocks, so we should have identified and notified customers about users with invalid email addresses before the rollout.
Problem 2: Multi-Account Access Challenges: Enterprises often have subsidiaries, each with its own domain name. As a result, users might log in to Snowflake using their parent company email address but use their subsidiary email address to log in to Bluesky. This, again, led to users being unable to access any data.
Lesson Learned: Multi-account access presents a complex challenge that requires further exploration. We are committed to finding a solution that facilitates seamless customer access management with complex organizational structures. We are actively exploring potential solutions, including establishing a more robust relationship with the individuals who manage our customers’ Identity Provider details and potentially re-evaluating our data model for how multiple accounts should be accessed.
Overall, this project embodied the typical experiences associated with developing a new feature. We encountered difficulties defining clear requirements, had to adapt to unforeseen challenges, tackled scaling issues, and experienced growing pains during the rollout phase.
If you’re looking for an opportunity to tackle challenges throughout the entire development lifecycle, from ideation to implementation, check out our open positions and join us: https://jobs.ashbyhq.com/Bluesky! We’re constantly searching for engineers who possess customer empathy and can contribute to our next big push.