In this sixth episode of Access Control, a podcast providing practical security advice for startups, Ben Arent chats with Dylan Stamat, CTO at VerticalChange. VerticalChange is a simple data system application for the Social Sector. Dylan is a seasoned technology and startup founder with 10 years of experience building and growing businesses. VerticalChange was an early adopter of running HIPAA-related workloads on AWS during the early days of AWS circa 2012.
Key Topics on Access Control Podcast: Episode 6 – HIPAA Compliance for Startups
- VerticalChange was founded to create impact for the social sector and help its agencies digitize manual processes.
- VerticalChange provides a solution that combines CRM, analytics, and dynamic form-building.
- Regulations like HIPAA, HITRUST, and FERPA are very strict, and agencies have to put in place many controls in order to comply.
- Startups in the healthcare space need to have someone who understands HIPAA and is willing to put the time in to write all the policies and procedures that need to be in place to meet security and privacy rules.
- Using a combination of CloudTrail, Auth0 logs, and Teleport logs, VerticalChange is able to create a log flow and see what people are doing within the application.
Expanding Your Knowledge on Access Control Podcast: Episode 6 – HIPAA Compliance for Startups
- AWS resources on HIPAA compliance
- Q&A: How VerticalChange Secures Sensitive Data Using Open Source Tools
- SSRF Attack Examples and Mitigations
- Teleport Database Access
- Teleport Getting Started Guide
- Teleport Access Plane
- Teleport Application Access
- Teleport Kubernetes Access Guide
Ben: Welcome to Access Control, a podcast providing practical security advice for startups. Advice from people who’ve been there. Each episode we’ll interview a leader in their field and learn best practices and practical tips securing your org. For today’s episode, I’ll be talking to Dylan Stamat, CTO of VerticalChange. VerticalChange is a simple data systems application for the social sector. Dylan is a seasoned technologist and startup founder with 10 years’ experience building and growing businesses. VerticalChange was an early adopter of running HIPAA related workloads on AWS back in the early days of 2012. Hi, Dylan. Thanks for joining us today.
Dylan: Hey, man. Great to be here.
The Early Days of Software Development
Ben: To kick things off, can you tell me a little bit more about your background and the early days of software development in Santa Barbara?
Dylan: Yeah. Santa Barbara has kind of a rich history in software engineering and startups. It was kind of the original Silicon Beach actually, before I think, Santa Monica took that away from it at some point. But you got a lot of people coming out of UCSB that were in electrical engineering, material sciences and computer science. A lot of big research projects coming out of there and turning into startups.
The VerticalChange Mission
Ben: VerticalChange was founded in 2012. Can you tell me about the problem it tried to solve?
Dylan: A few of us started VerticalChange — we were working at a consultancy for years and years together, and we wanted to do something a little bit more impactful. And we know a lot of people that work in the social sector that were still using paper files and organizing their — the agencies’ data on paper and in books. And so we wanted to see if we can come in and help digitize that in a kind of a new modern way.
Serving the Social Sector
Ben: And what is defined as the social sector? People aren’t familiar with it.
Dylan: Yes. In the social sector is, let’s say, mental health agencies, homelessness shelters, agencies that provide services for the public. We work with, let’s say, early childhood development, with the cities. We work with San Francisco and San Diego, for example. Either small agencies or really large agencies with thousands of employees.
Ben: What is the offering that VerticalChange offers for these agencies?
Dylan: Yeah. So we are — I like to describe it — it’s sort of a complicated product. I like to describe it as part CRM. So the agencies can manage their clients. And then also it’s a report builder. So you can kind of collect analytics on all the data in the system and then it’s a form builder. You’re able to generate dynamic forms like a Survey Monkey or a Qualtrics. So it’s kind of a combination of Salesforce plus Qualtrics analytics platform on top of it.
Ben: And I guess the thing that makes VerticalChange different is around compliance and HIPAA, which we have to focus in for this episode. Tell me what makes it different for using VerticalChange instead of these sort of other solutions?
Dylan: Yeah. So all these regulations like HIPAA, HITRUST, FERPA, they are very strict and they kind of come with a lot of controls you need to put in place as an agency to meet the control. It’s hard for agencies to do. If you’re a, let’s say, a five person agency that deals with mental health, you need to make sure that all your files are properly stored. You need to make sure you retain your files for a certain amount of time. You have really articulate offboarding and onboarding processes for your employees because it’s such a highly sensitive information. And those are really hard to kind of, number one, do in general, whether it’s digital or paper based.
Ben: And so I know HIPAA — I looked the acronym up. It stands for the Health Insurance Portability and Accountability Act. Could you just tell me a bit more about what exactly HIPAA means as a startup dealing with healthcare data?
Dylan: Yeah. So HIPAA in general is a bunch of security and privacy rules to protect individuals health information. That’s why you go to a doctor’s office and oftentimes there would be a check in form with a bunch of people who have been there before. And the leaking of that information can lead to a lot of security concerns and privacy concerns. And the ability to — and that comes with a bunch of technical constraints too, storing that data. There’s a bunch of HIPAA laws around how you store digital data. And that’s also with the HITECH and HITRUST. It goes up the scope of compliance, right? And then you start looking at laws in terms of the government where you drop the list compliance and CMMC. So this is a compliance level and it’s healthcare specific and it’s complicated.
Reasons for Selecting AWS
Ben: Yes. In 2012, AWS about six years old, which sounds quite old, but it still wasn’t that mature. Can you tell me about why you picked AWS at that point over more traditional options, such as having bare-metal servers and controlling them yourself?
Dylan: Yeah. We came from a consultancy that worked with Amazon for quite a while. I myself had been working with AWS since it was in alpha. So I was really comfortable with the cloud. It did bring some challenges in terms of HIPAA though. Going back in 2012, Amazon actually wasn’t signing business associate agreements with customers. And that is an agreement that as a covered entity in HIPAA, that when you work with business partners and you share sensitive data, it’s kind of a downstream effect of you sign these agreements. And so you can already create this web of trust of your data is safe. And so Amazon wasn’t actually signing those. So we were one of the first companies to eventually sign a BAA with AWS.
Ben: And does that fit into their shared responsibility model?
Dylan: Oh, gosh, yeah. It definitely left a lot for the implementers. So what we had to do is we had to, let’s say, encrypt our own disks with a certain encryption algorithm. EBS at the time we wasn’t doing disk encryption. We had to do a lot of things on the networking side that wasn’t really serving in that machine and do a bunch of kind of strict policies there. So there was probably a laundry list of things that we had to do that were hard and were not very easy. Amazon at the time didn’t make it very turnkey because that wasn’t their focus. But nowadays it would have been much easier to — and you can go onto a certain page on it you’d be watching it list all the services that are covered by HIPAA. You click buttons here to enable encryption everywhere. Back then we had to actually run on dedicated instances as well. And those were extremely expensive to run, especially as a startup.
Guidelines for Running HIPAA Workloads at AWS
Ben: But fast forward 2021, [inaudible] is much easier now to run HIPAA workloads on AWS. Are there any sort of features that you leverage now that make your life much easier?
Dylan: Yeah. I mean, we use encryption everywhere that comes out of the box from AWS. Each of our customers, we generate encryption keys for them using the key management service. And we have — we want to use Cloud HSM, which is the dedicated hardware at some point as well. Their tools around — kind of the networking side is very easy. The S3, you can store stuff there and have it encrypted using these keys. And Amazon provides a lot of resources as well where you can go and you can say, “Okay. Am I doing this properly? Are my machines up to date?” Using AWS Systems Manager. So, yeah, the services that Amazon provides have gotten so confusing, and there’s so many of them. But when you know which ones to use and you hand-pick the right ones, it actually ends up being a pretty nice situation.
Ben: Yeah. So if you were to build VerticalChange from scratch, would you still stay with Amazon, or would you look at other cloud providers?
Dylan: I would still stay with Amazon. Yeah. And you eventually want to get into the government sector. So we’re going to [inaudible] for GovCloud at some point.
Ben: And have you investigated the GovCloud at all or is still early on in the —?
Dylan: Yeah. We’ve investigated. We make sure that all the services we use are compatible with GovCloud. So when we do eventually go down that compliance path, it’ll be a very easy transition over.
Ben: Yes. You want to make sure that you’re not using something that isn’t available.
Dylan: Yeah. Exactly.
Ben: So you don’t have to rewrite —
Dylan: Yeah. Exactly.
Tips for Companies Entering the Healthcare Sector
Ben: So do you have any tips for other companies who are either thinking about having a healthcare startup or sort of pivoting to the healthcare domain as far as possibly, like, architecting or hosting their application?
Dylan: Yeah. I would say make sure wherever you are hosting your application on technical side, that you kind of go through a list of items that — wherever that might be. Amazon has a bunch of resources. GCP has a bunch of resources specifically for HIPAA and running on the on the cloud. If you’re going to bare-metal, then you’re going to go and you’re going to get some more antiquated documents that you’ll start to look at. But I think one of the main things when you’re creating a startup right now in the healthcare space is that you have somebody that understands HIPAA, even if it’s from a high level. And is willing to put the time in to write all the policies and procedures that need to be put in place to really meet these security and privacy rules. And it’s a really time-consuming process. Very similar to going through a SOC 2 Type 1. You had your business continuity plan in place. Use this type of encryption, whatever our password policy is. So it’s really an ongoing process. A lot of these are things that you want to practice as well. So annually you’ll do this or quarterly you’ll do this and keeping all your employees aware what HIPAA is, even if it’s of a high level and doing security trainings and stuff like that. So it’s a lot of personal stuff. Yeah.
Steps to Achieve HIPAA Compliance
Ben: You kind of touched on it. But is there sort of defined steps that are required to become HIPAA compliant or is it a case of your —? Does HIPAA go through in auditive review, or is it sort of just up down to your own paperwork and policies?
Dylan: It’s up to your own paperwork and policies. There is no governing board that will say, “Hey, you’re HIPAA compliance,” and stamp your company. So it’s different than, let’s say, a SOC 2 Type 1 where you have accredited compliance. So that is a little bit odd and a little fuzzy. And there’s a lot of people online trying to figure out what this rule means. So it could be really confusing. And it’s a lot of times using your best judgment and then when you don’t know about something, look for that rule and try to decipher it and find some documents about it. But there’s a lot of things that you kind of need to put in place or else you can get into a lot of trouble. If someone were to report you as having a HIPAA violation, you will get in a lot of trouble. And you will get fined a lot of money. Let’s say there’s rules where, let’s say, a certain amount of your customers’ data was leaked. Then you have to do a formal request to the Office for Civil Rights and Department of Health and Human Services. So there’s a bunch of steps you need to take when reporting. And you want to make sure you have those in order and the procedures in place. If that does happen, you don’t get caught not reporting yourself in time and then getting in a lot more trouble.
Ben: A slightly different take. We tried, in our past episodes, to talk about security incidents and people trying to run crypto miners on devices. That’s kind of the issue kicking people out. But I have seen recently, actually, the ransomware has had two prongs. One is they will encrypt your disk, but the second one is they will also say they have the data and they’re going to leak it.
Dylan: Yep. Yep. Yeah. And people are paying the ransoms in a lot of cases. And in a lot of cases, it makes sense to because they might not have a response to that technically or from the business point of view. Unfortunately, it’s led to some instances, I think it was in Germany, recently, where somebody was rerouting ambulances and there were some deaths involved. So it can be pretty bad.
Ben: Yeah. I saw so in Ireland too. The Irish Health Service also had a pretty bad ransomware attack too. And I think that was another case in which they were going to leak patient information. When the leak comes, it’s pretty bad for not only the company, but potentially the people whose information is going to leak.
Dylan: Yeah, yeah. It’s really scary. And there’s a page I think it’s in [inaudible] the ongoing list of HIPAA violations. So it shows line by line what company most recent and what the fee was. And some of these fees are huge. And often the entities that are getting these violations are colleges or hospitals. And so it’s all small agencies that you might be working with the homeless. And they often get hit by this huge HIPAA fine because the agency didn’t know. It’s really bad and it really sucks for them. And that’s why it’s kind of important just to have, I think as a startup, just to keep everybody in the loop and always reiterate the importance of HIPAA. And then also as a company, what we’re trying to do is really kind of be privacy and security first. So any kind of customer that we work with, we figure out we’re signing BAAs with them. So we kind of figure out, “Hey, what are you guys doing on your end for HIPAA? How do you do this? What do you do?” And it really kind of sharing information and being really allies with our customers. Because nobody wants to get in trouble and everyone wants to keep growing and secure their privacy.
Encountering Teleport and How It Changed Workflow
Ben: You’re also one of the first Teleport users on the podcast.
Dylan: Oh, cool.
Ben: And I’m sort of interested about how you heard about Teleport and how has it changed your workflow and whether it helped you on the sort of HIPAA compliance journey?
Dylan: Yeah. So we had our original v1 architecture that we started way back in the day. And we, think about a year and a half ago, migrated to a new infrastructure. The new infrastructure is just kind of completely different than the old. And now we’re running on Kubernetes for our applications. A lot more service-oriented. And then we use Auth0 for identity management and authorization and authentication. And so we wanted to — from the kind of top of the stack to the bottom of the stack. Make sure that we have really detailed audit logs for anything that happens from any employee or any end user. From a combination of using CloudTrail, using Auth0 logs, using Teleport logs, we’re able to create a pretty cool log flow and see what people are doing within the application. If a medication changed that they were tracking in our application six months ago, we want to know who changed it and have all the details around that. So Teleport’s really helped kind of from that side of the auditing side but also the access side of being able to, in a role-based manner, provide access to different parts of our infrastructure, to different types of employees.
Ben: And do you use it for Kubernetes access as well if anyone needs to get — use [inaudible] any sort of debugging?
Dylan: Yep, yep. Exactly. Exactly. So all the engineers and, yeah.
Other Applications Used and Recommended
Ben: The last guest — we talked about log aggregation. What do you use to collect all of these logs and sort of run analysis on what’s happening?
Dylan: Yeah. We use Elasticsearch and Kibana. We also use S3 and Athena. And the logs — there’s a lot of logs. So back in the day, I think, you’d run the application if you’re just a normal startup. You have your logs and you’d only keep them for a little bit of time. But with having to do the HIPAA laws and trying to really keep a trail throughout the stack, especially when you add to Kubernetes, which creates a lot of logs from standard out. We’ve got huge Elasticsearch cluster now. Way bigger than a normal company would have.
Ben: And do you have a certain policy for how long you plan to keep them or archive them?
Dylan: Yeah. We generally keep logs for seven years. I believe that’s the number, seven years as one of the privacy rules. So seven years. But contractually, customers can change that on a case by case basis if they to.
Ben: What other applications —? Do you have any other recommendations for startups could use?
Dylan: I mean, if you’re on the database side, I would highly recommend RDS or anything that you don’t have to — that does encryption [inaudible] by default. It’s kind of out of your way that saved us so much time just managing that.
Ben: Well, I’m guessing do you shard per customer, or you have a database per customer as well or did more isolation?
Dylan: Yeah. We originally started, and we were doing isolation in a couple of different ways. We don’t shard per customer. But what we do is we use role-level security in Postgres. Which is pretty cool because you’re allowed to basically take your own encryption keys and make sure that only this agency can touch the data in this table at role level. And it’s really nice to be able to do that versus using, let’s say, a separate schema per customer. Because doing a type of analysis where you need to look across customers is going to be extremely hard when you’re doing different schemas. And so having everything kind of be in one data set. But using role-level security is a pretty amazing thing to be able to do.
Top Security Concerns
Ben: So what keeps you up at night? What’s your biggest security concern you currently have?
Dylan: Oh, man, there’s a lot of them [laughter]. How do you know they kept me up at night? A lot of the things that I deal with is — and I think some of the infrastructure part is accessibility to data. So it’s like what the CIA triad — is confidentiality, integrity and availability. And so that’s kind of the three pillars of data security. And availability is a huge one. Running Kubernetes, which is great. It’s a fast moving project. And things come up and we want to make sure that data is available to our customers at all times. And we have a lot of cases where that’s really important. For instance, I always use a case of mental health. If an agency gets a call from a suicidal customer or client, they need to be able to look that data up about that client as soon as possible. And prior to being digital, they’d run over to a filing cabinet, look under the letter of their last name, pull that up. And so we want to be faster than that, right? You always have a computer anyway. So let’s just quickly find it. But if it’s not there or you’re having some downtime, that’s kind of a life and death situation.
Security Measures Taken
Ben: That’s kind of around availability. As far as other security concerns with other team members, do you do anything for phishing, training or sort of how do you get your employees sort of trained on these sort of other controls that you have as part of your typical workday?
Dylan: Yeah, that’s a good one. We do kind of annual security reviews and then anybody that onboards we use Wizer, W-I-Z-E-R tech. They do a great video series of just general security. And then also HIPAA, they cover HIPAA. And they have a good phishing tool so you can phish your employers, which is pretty fun. I have a lot of fun with that. And a lot of little emails that go out just as reminders to employees. And then on the annual reviews we use — PagerDuty has a great security review for their employees and their engineers kind of split into two camps. And so if you ever want employee security [inaudible] or kind of slideshow to go through, it’s about an hour long. Look for your Pager Duties. And it’s amazing. It’s funny. It kind of covers every single topic you could think of.
Advice for Tech Experts Seeking to Make an Impact
Ben: As suppose sort of closing out here. For anyone who is — start in the same journal yourself, you kind of like went from a technology field into healthcare startup because you wanted to be more impactful. For other people on that journey, what would be your advice to them?
Dylan: I would just be really aware of HIPAA and what the whole healthcare ecosystem and how companies that are in it act. And then also I would go and maybe as a first step, do a SOC 2 Type 1 and just see what’s involved in that process. And you’ll realize how far you have to go into documentations and policies and procedures and just how nuanced it is. And so that’s kind of would be your first — and it also gets your business in a nice place on SOC 2 Type 1. It really covers kind of — it makes your company very attractive in terms of how put together you have your — your ducks in a row.
Ben: And Type 1 is self-validated.
Dylan: Well, Type 1 you actually have to get a third party to validate it. And then Type 2 is kind of an annual one you’ll do, which is a lot less — it’s kind of your annual update of it. So you do the first one and then you do a little updates afterwards. And then also I think the other piece of advice would be to, yeah, just your every employee that you have really just ingrain security as part of your company’s ethos. Security, security, security. You can have anybody call up and do social engineering against your employees. If you provide phone support or email support. Don’t email around documents that have PII or PHI. And just really basic stuff that you just don’t — most people wouldn’t think of. In any of the other jobs even if it was a software company that they worked at before, they might be so accustomed doing certain things, certain ways. And they just need to really change the way they do things. You can’t just share a word document with another customer that has PHI in it. You need to do a lot more — there’s a lot more involved steps to make sure that data is secure.
Ben: So I think to kind of eyeball it down is — for anyone who’s getting into this realm, it’s sort of like getting into finance but you always have your taxes. And there’s always a lot more paperwork and admin to do. But lots of fun, interesting technology challenges for people who I guess really need it.
Dylan: Yeah, yeah. Definitely. I mean, every day I’m working and I’m like: “Okay. Cool, this is actually really helping our customers.” And sort of it’s a good feeling that you’re helping out.
Ben: Well, thank you, Dylan, for your time today.
Dylan: Awesome. Thank you, Ben. I appreciate it.
Ben: This podcast was created by Teleport. Teleport allows engineers and security professionals to unified access for SSH servers, Kubernetes clusters, web applications and databases across all environments. To more, visit us at goteleport.com.