Chat with any video.
Paste a link from YouTube, Facebook, TikTok, or X — or upload your own clip. Viqu turns it into a knowledge index you can talk to, with answers cited to the second.
videoquery.site/app/videos/8f3c…
02:1405:08
00:42Welcome to the deep dive on transformer attention…
01:17Each head learns a different relationship pattern…
02:08Here we visualize the attention matrix as a heatmap.
Chat
What did the host say about positional encoding?
Around 03:21, they explain that sinusoidal encodings let the model generalize to unseen sequence lengths.
Ask anything…
How it works
From URL to conversation in under a minute.
1. Drop a video
YouTube URL, any web link, or direct upload up to 4 GB.
2. We extract everything
Transcript, scenes, on-screen text, visual context — all indexed.
3. Ask anything
The assistant answers with timestamps that scrub the player.
Ask a real video a real question.
Try YouTube, Facebook, TikTok — or add any other source. Same cited answers, to the second. Pick a demo below, or bring your own from a web link or upload.
I was laid off by Atlassian
Vasilios Syrakis · YouTube · 40:70
Sample: “I was laid off by Atlassian” by Vasilios Syrakis — played via YouTube embed.
View originalTranscript
- 00:00I was recently affected by the layoffs
- 00:03made by Atlassian and I wanted to take
- 00:06some time out to reflect on the time
- 00:09that I spent working for Atlassian. I
- 00:11worked there for about eight years.
- 00:13During that time I built a lot of things
- 00:16and I wanted to talk about what I built,
- 00:18mainly the things that I personally
- 00:19found interesting or that I'm proud of.
- 00:21I hope that this video will be useful or
- 00:24helpful to someone who perhaps is or was
- 00:28in the same situation as me and maybe
- 00:31it'll give them some inspiration in
- 00:32terms of how they can tackle the same
- 00:35things that I did or something similar
- 00:37and perhaps avoid some of the mistakes
- 00:39that I've made. I also might talk about
- 00:41non-technical parts of my experience at
- 00:43Atlassian, although most of it will be
- 00:45technical and this video will be split
- 00:48into chapters so that you can skip to
- 00:51sections that are more interesting to
- 00:54you rather than rather than watching the
- 00:56video from start to finish. So I suppose
- 00:58to start with I'll talk about when I was
- 01:00first hired and even though I it was
- 01:02eight years ago, I still remember the
- 01:04interview process, which is different
- 01:07nowadays, and the reason why I was hired
- 01:10or at least from my perspective the
- 01:11reason why I was hired and the things
- 01:13that I started working on during the
- 01:15start. So yeah, let's just start at the
- 01:18interview process. So I was interviewed
- 01:20by some people that I now consider
- 01:23friends and I remember having the
- 01:25impression while being interviewed that
- 01:28these individuals were quite intelligent
- 01:30and that was something that was exciting
- 01:32for me. The interview process consisted
- 01:33of a coding quiz on HackerRank, which I
- 01:36aced with full marks. Then the first
- 01:39technical interview was with two
- 01:42interviewers and they gave me a white
- 01:44paper and asked me to read it while they
- 01:46sat out of the room for about 10
- 01:48minutes. They came back in and then
- 01:50asked me questions about the white
- 01:52paper, asked me to basically articulate
- 01:54what was in in white paper and the white
- 01:56paper was actually about custom domains,
- 01:59and the white paper was by Cloudflare.
- 02:01They then asked me a few questions about
- 02:03things like microservices and
- 02:05architectural things like that, um
- 02:07containers and and whatnot. And they
- 02:09were happy enough. I don't remember the
- 02:11rest, but they were happy enough with
- 02:13me during that stage, so I
- 02:15continued to the uh second technical
- 02:18interview, which was a troubleshooting
- 02:19exercise where I was asked to
- 02:22essentially prompt the interviewer for
- 02:24information in order to troubleshoot a
- 02:27real incident that occurred in
- 02:28Atlassian. And it was a it was an
- 02:30application problem that lead led to a
- 02:34denial of service. Uh so that was fun.
- 02:36And then I think I was asked something
- 02:38about how um latency-based DNS works,
- 02:42and my answer was not accurate, but
- 02:45perhaps acceptable. I I I thought about
- 02:48it from first principles, and I thought
- 02:49that that's uh for example, I thought
- 02:52that Route 53 did a triangulation based
- 02:54on the actual latency of the client, but
- 02:58it is more like that they use a uh they
- 03:01probably use a geolocation database in
- 03:04order to do latency-based routing of DNS
- 03:07requests of the DNS answers, sorry. Then
- 03:10after that was a values uh interview.
- 03:12And to be honest, I don't really
- 03:14remember most of the questions for for
- 03:16the values portion, but
- 03:18I do remember one thing, which was when
- 03:21I asked the question of I asked I asked
- 03:24the interviewers to think about 12
- 03:26months from now and to look back
- 03:28retrospectively, what is the thing that
- 03:30I would have had to achieve in order to
- 03:33for for you to say it was a good
- 03:35decision hiring this person. And then
- 03:36they told me about
- 03:38a [clears throat]
- 03:39an application that they needed to be
- 03:41built for the platform within Atlassian,
- 03:44and the application would facilitate
- 03:47self-service load balancers. Sort of
- 03:48similar to if you were using Amazon
- 03:52application load balances or the
- 03:54equivalent in any cloud provider. But
- 03:55for the internal developers of
- 03:58Atlassian. And it was essentially just a
- 04:00framework that I personally was not
- 04:03familiar with. And I said I could build
- 04:05it because I had confidence in building
- 04:07web apps with Python at that time. And
- 04:10they accepted my level of confidence and
- 04:13decided to hire me. So that's the the
- 04:14interview portion completed. So I joined
- 04:17Atlassian and they have this classic
- 04:20saying or impression that when you join
- 04:23Atlassian that you are drinking from the
- 04:26fire hose because there's so much
- 04:28information that you have to absorb in
- 04:30the first few weeks and months in order
- 04:32to just sort of get going. My first my
- 04:35very first task that at least task that
- 04:38I gave myself was to build the
- 04:40application that they had told me that
- 04:43they wanted. Let me just open a browser
- 04:45and we'll take a little looky at what I
- 04:48mean exactly. Let me just uh move my
- 04:51face a little bit. There's more real
- 04:52estate. Now obviously it's scalar draw
- 04:55is not what I care about. So they wanted
- 04:57me to build an open service broker. This
- 04:59is
- 05:00a a web app with an API which
- 05:02facilitates the provisioning of
- 05:04resources for a platform essentially. So
- 05:07you can you can see here
- 05:09it's sort of built to operate in a
- 05:11Kubernetes world where you're submitting
- 05:13these provisioning requests as things
- 05:17come up and down. And it's going to bind
- 05:20a resource to your pod or your cloud
- 05:23instance or whatever it is as you can
- 05:25see here. And it sort of sits in between
- 05:28these real resources. So you might
- 05:31provision something like a database and
- 05:34then you'll get MySQL. So you'll get
- 05:36something that's SQL compatible but
- 05:38that's abstracted away for your internal
- 05:40developers. Anyhow,
- 05:42the spec is uh, is here on GitHub. You
- 05:46can take a look at it. It goes into It
- 05:48goes into, like, for example, the
- 05:51catalog endpoint. And the catalog lists
- 05:54all of the services and plans that are
- 05:56available on the OSB, and, uh, just
- 06:00metadata about them. And you might say
- 06:03query the the service broker and then
- 06:05display some of the metadata in your
- 06:08Maybe you've got a a a console a
- 06:10console, like, the Amazon console, but
- 06:12maybe you've got something like that
- 06:13internally. Where developers can click
- 06:16and provision things. In the Atlassian
- 06:18case, it was all through configuration
- 06:20files that were committed to, um,
- 06:23version control, and then those would be
- 06:25uploaded, uh, during uploaded from a a
- 06:27build server to deploy a service. Um,
- 06:30but, yeah. So, you might have, you know,
- 06:32other APIs like, uh, provisioning here.
- 06:35So, put and patch for updating and
- 06:38deletes and blah blah blah. So, you
- 06:40would just basically go ahead and and
- 06:41implement this. Or
- 06:43I mean, if you wanted to build your own,
- 06:45and that is essentially what I did. You
- 06:47can see also there's a an open API
- 06:49document here that has the endpoints.
- 06:52So, I chose to build this in in Python
- 06:54using Flask. Oh, no. In fact, what I
- 06:57What I built it with first is with a
- 07:00library called Connection. This is a
- 07:02Python library which takes an open API
- 07:05document and then creates the API
- 07:09handlers for the paths for the API for
- 07:12the API routes that are in that
- 07:13document. Which is cool, but then we
- 07:15eventually I eventually migrated that to
- 07:18just pure Flask. And then eventually
- 07:21migrated that to, uh, Fast API, which I
- 07:24believe is what it still is at the
- 07:25moment. Um, okay. So, it's the first 2
- 07:29weeks, and my primary focus is to build
- 07:32sort of what I promised in the
- 07:34interview, which is this web app that's
- 07:36going to be a broker for the platform,
- 07:39and is going to allow self-service
- 07:41provisioning of load balancing in
- 07:44Elastic. So, like I mentioned earlier, I
- 07:46started with this library called
- 07:48connection, which took an open API
- 07:49document, turned that into routes. But,
- 07:52I'm going to just go with
- 07:53what it ended up as, which is a fast API
- 07:56app. Let's just say we've got fast API
- 07:58here, and then we've got
- 08:02a worker, and then we have a database,
- 08:05which was DynamoDB. Oh, that's annoying.
- 08:09And we would have a client making
- 08:11requests. That's why that's a fast API.
- 08:14The client would say, "Hey, please
- 08:16provision something for me." And the web
- 08:19worker wouldn't do it itself. It would
- 08:21actually send that over SQS. It would
- 08:25drop the task details into SQS.
- 08:28And the worker would then handle that.
- 08:31So, what does a provisioning task
- 08:33actually look like?
- 08:35It's something like creating
- 08:38DNS records somewhere, maybe creating a
- 08:42CloudFront distribution, maybe creating
- 08:47some API calls. And this would be the
- 08:51provisioning task that the worker would
- 08:53do asynchronously, while the web and
- 08:57client would wait for it to be completed
- 08:59essentially. So, the client's polling
- 09:01continuously to say, "Is it ready? Is it
- 09:03ready?" And when it is completed, the
- 09:06worker writes it to the database, the
- 09:08web server checks the status, and then
- 09:10responds saying, "Yes, it's finished."
- 09:12Or it'll say that something went wrong
- 09:15and there was an error.
- 09:16So, then we can sort of encapsulate this
- 09:18as the open service broker that I built.
- 09:23Pretty straightforward.
- 09:25Um to be honest, there's not much more
- 09:27to this, but we're going to go and talk
- 09:30about some of the more complicated bits
- 09:33in just a second, and I will directly
- 09:35link to this as well. So, we got this
- 09:37client requesting
- 09:39uh let's say "Please provision a load
- 09:41balancing." And that is essentially what
- 09:43they were asking for was some kind of
- 09:46load balancing somewhere in the edge
- 09:48infrastructure of Atlassian to allow
- 09:50traffic to go to their service. So,
- 09:53that's a good uh demarcation point to
- 09:56start talking about the next thing that
- 09:58I sort of built.
- 09:59And I built it through necessity
- 10:02of Essentially, I began to understand
- 10:06and unravel the requirements more as I
- 10:09went along. One of the architects had
- 10:11this idea to replace the load balancers
- 10:15at Atlassian, which were enterprise load
- 10:18balancers that had licensing costs, with
- 10:21a open-source cloud-native sort of
- 10:24commodity proxy. And the tech that we
- 10:26chose for that was Envoy proxy. You may
- 10:29be familiar with Envoy proxy. If you're
- 10:31not, then it's very similar to something
- 10:33like Nginx, but perhaps more modern than
- 10:37Nginx. Um you can take a look at its,
- 10:40you know, uh what's what's great about
- 10:42it if you want. You can just read
- 10:44through like why why choose Envoy, blah
- 10:46blah. But essentially, we wanted
- 10:47[clears throat] to replace the
- 10:48enterprise load balancers we had, make
- 10:50them self-service, so that devs
- 10:52effectively didn't have to talk to us to
- 10:55go set up their load balancing. So,
- 10:57Envoy has an API that allows you to
- 11:02configure it dynamically. Being able to
- 11:04reload the configuration at run time
- 11:06means that you can deploy a whole bunch
- 11:08of proxies and have them sit there
- 11:11running all the time.
- 11:12And then when someone needs different
- 11:14configuration for their particular uh
- 11:16service, then they can push out a change
- 11:19through the provisioning task detailed
- 11:21here. And those changes should flow to
- 11:25the proxy somehow. And so now, that's a
- 11:27good time to talk about the Envoy
- 11:30management server that I built, which we
- 11:32called the Envoy control plane. And this
- 11:35was it's essentially quite similar
- 11:40uh to this.
- 11:41Yet again, we used a Fast API app. But
- 11:44this was slightly different actually.
- 11:46Let's go into a little bit of detail
- 11:47here. I'm just going to wing this
- 11:49because I should be able to wing it
- 11:50because I know it quite well. Uh this is
- 11:52actually a I open sourced this this
- 11:55software and I called it Sovereign. You
- 11:56can actually go find that on Bitbucket.
- 11:58It's it's a public repo at least for
- 12:01now. I don't know if that's going to be
- 12:03the case always.
- 12:04But essentially Sovereign runs a Fast
- 12:06API app. And some of the things that it
- 12:09takes in as
- 12:11uh configuration are templates
- 12:15and context. And so the app uh polls
- 12:20these. Uh it's obviously got like uh say
- 12:24let's just say this is the
- 12:25configuration. Okay, let's stick Now,
- 12:28so the templates might be particular
- 12:32resource types. And in Envoy, you've got
- 12:35stuff like clusters, routes, listeners.
- 12:39And let's just leave it at that for the
- 12:40moment. You'd have these kind of
- 12:43templates. And so when this when this
- 12:47management server loads up, it'll read
- 12:49in these templates in the context and
- 12:50make these available as APIs for the uh
- 12:55proxies. So then you can imagine let's
- 12:58just say we've got uh an Envoy here.
- 13:02It is going to request these things and
- 13:05Sovereign is going to respond by taking
- 13:08the context, putting it into the
- 13:10templates, and rendering out different
- 13:12content uh as the context changes. Now,
- 13:15where does the context come from? Well,
- 13:17this is part of this management server
- 13:19that is dynamic. So well, let's just uh
- 13:22let's just do a bit of flip around here.
- 13:26Put the context
- 13:27actually comes from this
- 13:30database, but we are requesting it from
- 13:34the broker. So, the we're we're
- 13:36requesting data from the broker and
- 13:38other sources, in fact. Let's just add
- 13:40another source here. Let's just say we
- 13:42have a little S3 bucket with some data,
- 13:45and maybe that data is changing over
- 13:46time. So, we take that data, it's
- 13:48dynamic, we feed that into the
- 13:49templates. The The templates have logic
- 13:53that spits out particular Envoy
- 13:56configuration, and then the proxy
- 13:57changes over time. So, what happens is
- 14:00we've got a client that's making a
- 14:02provisioning requests to our broker. The
- 14:04worker is doing some provisioning tasks,
- 14:08and then writing the new data to the
- 14:10database. Then the the management
- 14:13server, let's say. Stop this. Let's
- 14:15encapsulate this a little bit. The
- 14:16management server is then polling that
- 14:18data from various places and generating
- 14:21new configuration. That configuration
- 14:23hits the proxy, and then it starts doing
- 14:26different stuff. That is essentially the
- 14:28second part of what I built. So, we've
- 14:30got a broker, we've got a management
- 14:31server, we've got the client, we've got
- 14:33the proxy. Uh why did this detach?
- 14:35Anyway, so now we've sort of figured
- 14:37this stuff out.
- 14:39This is all at a very high level.
- 14:41>> [clears throat]
- 14:41>> So, we've got this created. Now we can
- 14:44sort of think more about more
- 14:47infrastructure type things. We've got
- 14:49this proxy, but how do we end up with
- 14:51this proxy? How does that actually get
- 14:53provisioned? What is it? Where does it
- 14:55live? Well, let's start with one thing,
- 14:58which is that these proxies,
- 15:02there's many many many of them, as you
- 15:04would expect, and they are provisioned
- 15:06by
- 15:07um
- 15:08they are provisioned by a CloudFormation
- 15:11template. This is an AWS thing that
- 15:13allows you to essentially do
- 15:15infrastructure as code, and it allows
- 15:17you to create resources in in AWS that
- 15:21you would normally create via the
- 15:23console if you were just uh uh let's say
- 15:26uh basic user. So, what kind of stuff do
- 15:29we create in here? Well, if we were to
- 15:31do this stuff from scratch, we'd
- 15:33probably have like a VPC and then we'd
- 15:36have uh you know, a subnet inside that
- 15:40VPC and maybe we'd have
- 15:43an internet gateway, maybe we'd have
- 15:47uh
- 15:47we'd all security group, maybe we'd have
- 15:50a key pair, maybe an IAM role.
- 15:54Um oh, of course we need to have the
- 15:56auto scaling group.
- 15:58Of course, that's what's going to be
- 16:01creating these
- 16:03EC2 instances.
- 16:05And well, the auto scaling group needs
- 16:10an AMI, doesn't it? Well, indeed it does
- 16:12need an AMI. IAM role has to be attached
- 16:15to
- 16:16uh must be attached to all these. The
- 16:18key pair goes on on the these. Security
- 16:21group is attached to to the
- 16:24this Well, it's probably attached to the
- 16:26auto scaling group to be fair. Well, the
- 16:28EC2 instances would inherit it from the
- 16:30ASG, blah blah blah. So, we've
- 16:33got all these like uh blocks of
- 16:35resources and stuff like that. Cool.
- 16:37Let's let's put these up put these up
- 16:38together, blah blah blah. Cool. So,
- 16:41yeah, we've kind of got like a little
- 16:42template going on here and it's creating
- 16:44these proxies in many different regions.
- 16:48Uh we might have we might have like
- 16:50uh an NLB in here, a layer four proxy.
- 16:54Maybe we'd have uh bit of maybe a bit of
- 16:57ACM. Of course, these acronyms might
- 16:59mean nothing to some people, but for
- 17:01people that have used AWS, they would
- 17:03know what these things are. And they
- 17:04know it's not really that complicated.
- 17:06It's like pretty basic building blocks
- 17:08and this is what we created
- 17:11uh
- 17:12say 2,000 proxies,
- 17:14uh something like 13 regions, blah blah
- 17:16blah. Um and we also had a little bit of
- 17:19route 53 records for other stuff. Now,
- 17:23the AMI, it's not really provisioned by
- 17:26the the template. It's more like it's
- 17:28referenced by the template, isn't it?
- 17:30So, that would bring us on to the next
- 17:32piece of this thing that I built, which
- 17:34is, well, we need to produce an AMI. We
- 17:37need to produce a standard image for
- 17:39these proxies, and it's going to include
- 17:41all the important stuff in there. So,
- 17:43how do we create this image for the
- 17:45proxy? Well, in this case, we had uh
- 17:48repository that was using HashiCorp
- 17:50Packer, and
- 17:52uh we had um
- 17:55a Salt Stack
- 17:56uh let's call it configuration.
- 17:58And so, we would use Packer to um let's
- 18:02say we'd have the EC Oh, we'd use the
- 18:05EC2 provisioner. And so, we would create
- 18:08an EC2 in like a dev account. We'd then
- 18:11upload all of our Salt Stack
- 18:12configuration. Salt Stack, by the way,
- 18:14is very similar to Puppet, Ansible, and
- 18:16Chef, in case you're not aware of what
- 18:18those are. It is configuration
- 18:19management tools, and that's a fancy way
- 18:22of saying that I want to run I want to
- 18:25install packages, put files, and run
- 18:28services on a machine in a particular
- 18:30way, in a particular order, and it
- 18:32automates that process, makes that
- 18:34process declarative for you. Well, not
- 18:37for you, but it helps you to to make it
- 18:38declarative. So, we created a little um
- 18:41created a little EC2 live running EC2
- 18:44here. We dump the config on there, we do
- 18:47a provisioning step, and then we take
- 18:49the
- 18:50uh essentially turn this into an image,
- 18:52like shut it down, uh whatever, snapshot
- 18:54it, and turn it into an image. So, that
- 18:56essentially would just produce this uh
- 18:58AMI. Now, what was included in here?
- 19:00Let's Let's say we can just uh we can
- 19:02include a few things here. Let's just
- 19:04say we had um we had states for for
- 19:07Envoy. So, like let's say install,
- 19:10configure,
- 19:11uh let's say just install and configure
- 19:13Envoy Uh
- 19:15logging agents, security,
- 19:18let's say slash hardening, network
- 19:19tuning,
- 19:21containers,
- 19:23tracing. Oh, let's just say let's just
- 19:25say observability agent there. And that
- 19:28can cover I can cover logging, tracing,
- 19:30metrics. So, that's essentially what's
- 19:32going on here. Produces the AMI,
- 19:35CloudFormation template takes this AMI
- 19:37provisions these EC2s
- 19:40EC2. And they're running with all this
- 19:42stuff. And then when they when they get
- 19:44provisioned, that's something that we
- 19:46forgot here. There's parameters.
- 19:48Parameters, bump that up, bump this up,
- 19:52make it neat. So, we've got the
- 19:53parameters and these at
- 19:57runtime would pass in secrets and keys
- 20:00and blah blah blah. And then these these
- 20:03proxies would grab the resources, be
- 20:05configured, and then they would be
- 20:07running and accepting traffic. Boom,
- 20:09that's it. Everything's done, working.
- 20:11This was essentially the first two years
- 20:14of working at Lyft. So, now when a
- 20:16developer says, "I want to run my
- 20:19service and I want it to be
- 20:21accessible on the
- 20:22internet with all the fancy bells and
- 20:24whistles and routing and advanced
- 20:25stuff." We'd say, "Yes, no problem. Let
- 20:28me just get that provisioned for you."
- 20:30We
- 20:31send off the provisioning task, we write
- 20:33something to the database, we tell them
- 20:34it's ready, then the management server,
- 20:37it's the
- 20:39broker says, "What's the current state
- 20:42of things?" Takes that data, plus other
- 20:44data, puts it into the templates,
- 20:46creates resources out of those
- 20:48templates, gives them to Envoy on when
- 20:50it requests them. And this was all
- 20:53pre-provisioned.
- 20:55That's long-lived infrastructure with
- 20:57CloudFormation. And the CloudFormation
- 20:58is relying on an AMI that it can use to
- 21:01provision those images, those machines.
- 21:03So, yeah, that is probably the first 24
- 21:06months. So, what was next after this?
- 21:08So, this was the foundation of our team,
- 21:10essentially the product that we were
- 21:12going forward with, um which is uh
- 21:14centralized load balancing managed by
- 21:17our team, and all of the features that
- 21:20we provided to our customers would live
- 21:22in logic defined in these templates.
- 21:25We've now laid the foundation for the
- 21:27team. We've got proxy infrastructure
- 21:29that's reacting dynamically to services
- 21:32that are being deployed with different
- 21:34configurations over time. What was next
- 21:36after that point? The big thing after
- 21:38that was taking some of the larger
- 21:40products and making it possible for them
- 21:43to use this platform component. That was
- 21:46one big part, and the second big part
- 21:48was migrating all of the microservices
- 21:51within Atlassian to use this. And that
- 21:53was relatively easier because we could
- 21:56enforce that through the platform.
- 21:58Essentially, what that means is that the
- 22:00platform was previously providing very
- 22:03basic load balancing to every service.
- 22:05And they forced a switch to where you
- 22:08could no longer expose your service
- 22:10publicly through their load balancer,
- 22:12which is too basic, and you had to go
- 22:14through our centralized load balancing
- 22:16infrastructure and to explicitly
- 22:18configure it as a way of signaling your
- 22:21intention for that service to be
- 22:23publicly accessible. Whereas previously,
- 22:26it could have just been maybe accidental
- 22:28that your service was public and not
- 22:30very well protected. So, that was the
- 22:32big major push. We got products like
- 22:35Jira, Confluence, Bitbucket, Status
- 22:38Page, and many others behind this edge
- 22:41infrastructure. And then, what was after
- 22:43that? Well, now we can sort of talk more
- 22:45about Let's say we can talk more about
- 22:48the uh the Envoy-based product that we
- 22:51had here. So, this particular thing,
- 22:54we've got this groundwork of being able
- 22:57to take basic inputs from a a developer
- 23:00and to turn that into templated
- 23:03configuration. Now, Envoy has a lot of
- 23:06configuration. It has a lot of stuff you
- 23:09can configure. Let's just look at the
- 23:11routes, for example. Let's look at the
- 23:13virtual host, for example. You can
- 23:15configure what domains to accept traffic
- 23:17on. Pretty basic. You can do routing.
- 23:19Sort of basic, but once you delve into
- 23:21how you can do this, it gets pretty
- 23:23complicated pretty quickly. You can
- 23:25match on different things. You can route
- 23:27it in different ways. You can do direct
- 23:29responses. do redirects, blah, blah,
- 23:31blah. You can add and remove headers. I
- 23:34guess I guess you could say that's
- 23:35pretty standard, but you can also
- 23:37choose, for example, when you're
- 23:39configuring a route action, you can also
- 23:40choose to send to any cluster that's on
- 23:44the proxy. So, then if I have a thousand
- 23:47devs, or a thousand services, and they
- 23:50each have their own cluster, and any
- 23:52route can send to any cluster, well, it
- 23:54sort of brings up this point of well,
- 23:56this
- 23:57data here needs to be validating that
- 24:01and abstracting that, and so on and so
- 24:05forth. So, there was definitely a
- 24:07concentration of a lot of the
- 24:09development work around this logic here,
- 24:11making sure it was validated here in
- 24:13terms of the parameters were validated
- 24:16such that when those parameters were run
- 24:18through the logic in these in these
- 24:21resources, that it would produce valid
- 24:23resources. Pretty standard, I suppose
- 24:25you could say. I don't know. Maybe I do
- 24:27feel like I have the curse of knowledge.
- 24:29Um and that this stuff seems easier to
- 24:32me now because I I've done so much with
- 24:34it. Uh but there's a lot. There's a lot
- 24:36in here. And if we go into, uh for
- 24:38example, extensions, there's a lot of
- 24:40extensions that can be applied to a
- 24:43listener or a or a cluster. For example,
- 24:46you might have, uh where is it? You've
- 24:48got, uh network filters here. You've got
- 24:51all kinds of network filters. And a big
- 24:53one that we obviously used was a HTTP
- 24:56connection manager, where you could
- 24:58configure routing and how to handle
- 25:01proxies and web sockets and all this
- 25:04stuff. And then, if we go a little bit
- 25:06before that, there's also things like
- 25:08external processing and external
- 25:11authorization. And this sort of brings
- 25:13us to Oh, let's say something that
- 25:16happened next. So, I did briefly mention
- 25:18that some of the big parts after
- 25:19building this was to migrate big
- 25:21products onto Let's assume that's all
- 25:23finished. It took It took some time. It
- 25:25took a couple years because there were
- 25:27many features that needed to be built
- 25:29out here and and wherever else in order
- 25:32to support the larger products and their
- 25:34special cases to work on
- 25:37what's effectively a generic
- 25:39multi-tenanted platform. So, let's just
- 25:41assume that they're all migrated. Then,
- 25:42we have more features that we want to
- 25:45add. I did sort of allude to like we
- 25:47have this We have this groundwork. We
- 25:49have this dynamic configuration. What
- 25:52I'm trying to say is that we we created
- 25:54opportunity. We created opportunity to
- 25:57centralize logic and to handle concerns
- 26:01early in the chain of requests. What I
- 26:04mean by that is a customer, let's just
- 26:07make a smiley customer. And customer is
- 26:10someone that's using our cloud products
- 26:12or Atlassian cloud products. They are
- 26:14hitting the
- 26:16Let's just say they're hitting an NLB
- 26:17first and that's then being proxied to
- 26:19these boys. Yes? If we can deal with the
- 26:22problems here before they reach a
- 26:24service,
- 26:26let's say and let's give it a square.
- 26:28Let's call it a back-end service, you
- 26:30know. So, the requests are flowing in
- 26:31from the customer to the proxies and to
- 26:34the Pretty standard stuff. If we can
- 26:35deal with certain concerns here before
- 26:39it reaches here, we save a lot of time,
- 26:41we save some money, which is and it
- 26:44saves the customer time. It's great for
- 26:45everyone, really. Um
- 26:47and one of those things Now, this is
- 26:49where the the diagram becomes
- 26:51complicated, so let's move off to the
- 26:53side. Let's just copy a few of these.
- 26:55Let's grab
- 26:56three things
- 26:58move over to the side. We've got the
- 26:59customer talking to the proxy and the
- 27:01proxy is talking to back end. Of course,
- 27:04the request comes back up and back out
- 27:06to the customer. Fine and dandy. Yes,
- 27:08this is a this is a proxy. Whatever,
- 27:10there's no surprises here. Now, without
- 27:12with with the products that Elysium
- 27:15runs, there's all kinds of stuff that
- 27:17needs to happen like authentication for
- 27:19example or authorization or
- 27:23DDoS protection or rate limiting or
- 27:26access logs. All this kinds of stuff
- 27:28that needs to happen and it's just turns
- 27:31out that we can deal with them here
- 27:32instead of on a bazillion bazillion back
- 27:37end services. Just imagine there are a
- 27:40bazillion bazillion of these. Just
- 27:43zillions upon zillions. See Daisy. Just
- 27:47zillions and zillions. They're like
- 27:48gazillion. Now, can you imagine if a
- 27:50thousand dev teams needed to deal with
- 27:53all this stuff plus more on their own
- 27:56service? It would be a tremendous waste
- 27:59of money for the company. It would slow
- 28:01down features. The customer wouldn't get
- 28:03their features when they need them and
- 28:05stuff is already hard enough to deliver
- 28:07as it is. Thus, the platform and
- 28:10centralized management of resources and
- 28:13implementation of these features. So,
- 28:16how how were some of these things
- 28:17implemented? Well, DDoS protection was
- 28:19really provided by
- 28:22CloudFront. That was
- 28:24that was
- 28:25spearheaded by a colleague of mine who
- 28:27is very smart and conscientious. And
- 28:30essentially, let's make this a bit more
- 28:33accurate. Let's just say let's get rid
- 28:34of these. There's an NLB here. Oh, blah
- 28:37blah blah. And of course, it's
- 28:39two-way. So, that's one way that we can
- 28:41take care of that concern for these back
- 28:43end services. Great, we've solved solved
- 28:45the concern for that. Fantastic. These
- 28:47others, well, access logs, what we can
- 28:50do is something like we can use these
- 28:53network filters. Yes, we use the network
- 28:55filters. For example, in the HCM, we
- 28:58have
- 29:03Where are the access logs? Access log.
- 29:05Now, remember, all of this configuration
- 29:08is dynamic. It's all dynamic and it is
- 29:11created by templates which abstract away
- 29:15the resource configuration from the
- 29:17developer who wants to configure it.
- 29:19They provide simple parameters. Those
- 29:21parameters are then validated and then
- 29:22they are fed into the template as
- 29:24context so that we produce the correct
- 29:27template. That means that
- 29:29they send us a little bit of JSON and we
- 29:32set up this whole thing for them with
- 29:34all the access logging and blah blah
- 29:36blah, whatever. So, that is done, in
- 29:38fact, inside the proxy, natively.
- 29:41Fantastic. Some of these things,
- 29:42however, a little bit more complicated.
- 29:44These things, we need to use a sidecar
- 29:47model where Envoy is talking out the
- 29:48side and then these are their own
- 29:51services running locally on the on the
- 29:53proxy. So, these would be like
- 29:56containers, essentially. We've got this
- 29:58sidecar model and those sidecars, some
- 30:01of them were contributed by other teams
- 30:03and some of them were created by me and
- 30:05our team. The authentication and the
- 30:07authentication sidecar was created by
- 30:09me, written, of course, in the Lord's
- 30:12language, Rust. Authorization was done
- 30:15by another team and rate limiting was
- 30:16done by another team. And so, they were
- 30:18able to contribute these sidecars,
- 30:20which, by the way, were set up and
- 30:23were downloaded and configured onto the
- 30:26AMI by this provisioning AMI
- 30:28provisioning flow. Great. So, now we
- 30:31have a programmable proxy with sidecars
- 30:35that have their own separate logic from
- 30:37the proxy and they, too, can actually
- 30:40receive configuration, which is dynamic
- 30:43over the wire locally and
- 30:46make it even more program. So, we're
- 30:48solving all these concerns before they
- 30:50hit these
- 30:51back ends and in very very little
- 30:54time. So, that was
- 30:55essentially that is some of the stuff I
- 30:58worked on after migrations and blah blah
- 31:00blah. Yay. What I do after that? With
- 31:03this big blob rid of this mess. So, then
- 31:05we had some non-technical requirements
- 31:08come through. More compliance
- 31:11and things like that. And that effort
- 31:13was very tedious and boring for me
- 31:15personally. It didn't involve building
- 31:17new stuff. It involved taking all of
- 31:19this, making sure that it was compliance
- 31:22for in certain ways. Very bored boring
- 31:26checklist ticking work. Blah blah blah.
- 31:29So, I said earlier that I would also go
- 31:30over some of the non-technical things
- 31:32that I had to go through while working
- 31:34at Atlassian. Obviously, all of that
- 31:35stuff is sort of high-level technical
- 31:37stuff that I just showed. What was some
- 31:40of the other stuff that I went through
- 31:42during my eight-year slog at Atlassian?
- 31:45The first few things to come that come
- 31:47to mind is that I have grown
- 31:50tremendously in my diplomacy skills,
- 31:54conflict avoidance, probably conflict
- 31:57resolution as well. Being able to
- 31:59persuade, propose ideas, being able to
- 32:02teach, educate, and mentor. These are
- 32:05the non-technical things that you
- 32:06probably don't hear a lot about. But
- 32:09after Another thing is that the ability
- 32:11to maintain things, maintain software
- 32:15and systems, to see where the cracks
- 32:17show up and to build things so that
- 32:21those cracks don't show up as or at
- 32:23least to make them show up late as
- 32:25possible. That's definitely something
- 32:26that I picked up. Let's just talk about
- 32:28that maintenance for a sec. I noticed
- 32:30over the eight years that I was there,
- 32:32when I built these apps, these
- 32:35services, that there's obviously that at
- 32:37the very start there's the requirement
- 32:40to onboard people and write
- 32:41documentation and train people so that
- 32:44they understand how things work, know
- 32:46how to contribute to them, and debug
- 32:48them. So that when they become when they
- 32:50go on call, they know where to look,
- 32:52what could go wrong, where do things
- 32:54break essentially. So, you know, that's
- 32:57whether that's knowing what kind of what
- 33:00particular log messages mean, what sort
- 33:02of metrics to check when something is
- 33:05going wrong and what those metrics could
- 33:07allude to, how to resolve those
- 33:10um you know, particular expected
- 33:12problems if they're not automated away.
- 33:14Um and this could be like, you know,
- 33:16Amazon could have an outage and the
- 33:18database isn't access for example. What
- 33:20do you do in that case? What if SQS
- 33:22stops working and you can't do any
- 33:24provisioning tasks? What how what impact
- 33:26does that have on the services that need
- 33:28to provision the resources? And how do
- 33:29you resolve um
- 33:31What happens if an if a proxy receives
- 33:33bad configuration? What if it receives
- 33:35configuration that's valid, but that
- 33:37destroys the traffic that's flowing
- 33:39through? How do you pick up on those?
- 33:41What do you check, etc. etc. So there's
- 33:43obviously a lot of that at the start
- 33:45when you build something. There's a lot
- 33:46of that at the start. But the thing
- 33:48that's more difficult is over time
- 33:50people come and go. People get hired.
- 33:53People get people leave for other jobs
- 33:55and whatnot. And so you get you have to
- 33:57do that onboarding again obviously. But
- 33:59you should have more people that are
- 34:01able to do that onboarding collectively.
- 34:02But then you sort of bring in new
- 34:04opinions. People look at an existing
- 34:06codebase and they want to change things.
- 34:08They want to make it better, and so on
- 34:10and so forth. And so they do that. And
- 34:12change ends There's I I suppose there's
- 34:15this concept of churn in the codebase.
- 34:18The area that churns, it's sort of it
- 34:20becomes predictable where all the churn
- 34:22is going to be
- 34:24at a certain stage. And once you notice
- 34:26that there is some churn, it's sort of a
- 34:29smell. It is it's an indication that
- 34:32part of the service or project is
- 34:35going to keep increasing in size or
- 34:38complexity. And something there needs to
- 34:41happen. Something needs to be done to
- 34:43avoid that mess. It's just just how
- 34:45software goes, I suppose. It'll be
- 34:47interesting with all these
- 34:49vibe coded apps and AI assisted apps to
- 34:52see how we handle that. When we have
- 34:54people that are not really familiar with
- 34:56what they've created, and the
- 34:58maintenance burdens appear. They don't
- 35:00appear at the beginning. There's just
- 35:01not enough going through. It hasn't been
- 35:03around for long enough. There hasn't
- 35:05been enough changes. Building something
- 35:06is easy. Changing it and making sure
- 35:08that it you can still change it over
- 35:09time is difficult. Because as you change
- 35:12things, it slowly becomes harder to
- 35:14change. Things start to get coupled, and
- 35:16all of a sudden when you change
- 35:18something in one area, it affects
- 35:19another, and you have to deal with the
- 35:20task of detangling something. And you
- 35:22might be able to find these areas quite
- 35:24quickly, get an LLM to perform the
- 35:26detangling for you. I think that's If we
- 35:28can do that, that's fantastic. But I
- 35:30don't want to be too optimistic just in
- 35:32case. So, there's that on That's my
- 35:35opinion on the maintenance side of
- 35:36things. The next thing I want to talk
- 35:38about is when I mentioned diplomacy,
- 35:41what I'm really trying to say is that I
- 35:43was exposed to different types of
- 35:46managers and colleagues over time. And
- 35:48everyone has different personalities and
- 35:50styles of working. And because I was
- 35:52exposed to so many different types, I
- 35:54experienced conflicts with certain
- 35:56people. And even though I had conflicts,
- 35:59there's still people that I respect.
- 36:00It's just something that happens when
- 36:03you when your personality doesn't mix
- 36:05with their personality. And that's just
- 36:06something that's a bit inevitable. And I
- 36:08think that the only thing you can really
- 36:10do in those situations is to try to have
- 36:13the self-awareness and the awareness of
- 36:15the other person and the, I suppose,
- 36:18understanding of psychology and and how
- 36:20people work to an extent, so that you
- 36:23can be responsible for that difference
- 36:25and the potential for conflict, and to
- 36:28handle it effective to to anticipate the
- 36:31conflict that's going to arise and and
- 36:33to do something to make the relationship
- 36:36work. And maybe it's impossible. I don't
- 36:37know. But that was definitely a source
- 36:39of great stress and at [clears throat]
- 36:41times it affected my performance. And so
- 36:45I do think that because it affected my
- 36:48performance that I took it quite
- 36:50seriously and I learned and changed as a
- 36:53result. So the next time that those
- 36:55situations come around, I do firmly
- 36:58believe that I'll be able to handle them
- 37:00quite a lot better. And then some of the
- 37:02other things, in fact one of the things
- 37:04that I found quite challenging was
- 37:08mentoring. And so I find it easy to help
- 37:11people to point out areas where they
- 37:15need understanding and to deliver that
- 37:18understanding to them, to break down
- 37:20complex things into simple terms so that
- 37:23they can build a mental model of the
- 37:26system that they're working on. I have
- 37:27that ability. I'm quite good at that.
- 37:29But mentoring is distinct from that. I
- 37:31had an intern in the last year and I
- 37:34will first say that the result of their
- 37:36internship was that they got the highest
- 37:38rating possible and it essentially
- 37:41guarantees
- 37:42an offer to work at a last year. The
- 37:44project that they worked on was very
- 37:46impressive and how they approached it
- 37:49and and built it was very impressive.
- 37:51And so that's why they got that
- 37:52excellent rating. What I found
- 37:54personally difficult was striking the
- 37:57balance between It was essentially
- 38:00striking the balance between how much
- 38:02time I give to the mentee and what that
- 38:06time would consist of, whether it's, you
- 38:08know, I didn't I don't want to give them
- 38:09answers to problems, but I don't want
- 38:11them to get so stuck that they become
- 38:13frustrated. I have no idea if I reached
- 38:16that balance, but I suppose the results
- 38:18speak for themselves. I I but I I don't
- 38:20I'm not sure if I can attribute
- 38:22the results to me necessarily. The
- 38:24intern was helped by some of my other
- 38:26colleagues when in areas that I'm much
- 38:28weaker in. So, they effectively got
- 38:31subject matter experts in a few
- 38:34different areas to contribute to their
- 38:36success. But then they did the majority
- 38:38of the legwork to actually build the
- 38:41thing and to test it and to make design
- 38:44decisions and stuff like that. And it
- 38:45was very successful. But I still have
- 38:46this lingering impression of feeling
- 38:48that mentoring is difficult for me and
- 38:51that I I don't have um a good way of um
- 38:55figuring that out because I've never
- 38:56been mentored myself. So, I don't really
- 38:58know what to expect and what they do.
- 39:01But I want to emphasize that that's a
- 39:03very specific type of mentoring that I'm
- 39:05not too sure about. Whereas training my
- 39:07colleagues, getting them to understand,
- 39:09working, you know, working through
- 39:11problems with my colleagues, that was
- 39:12essentially my bread and butter
- 39:14during the last half of my employment.
- 39:16You know, jumping on uh uh call and
- 39:19going through stuff. Feedback that I got
- 39:21from my colleagues all the time was that
- 39:22I was always available to help and that
- 39:24I could boil down hard topics into
- 39:26something that was understandable, which
- 39:29I'm pretty proud of. And I've been
- 39:31yapping for a while. I think that covers
- 39:32a quite a lot. If I remember more, I'll
- 39:34probably just make a second video. Um
- 39:36maybe maybe if people are interested, I
- 39:38could actually go through and build some
- 39:40of these things. I could actually go
- 39:41through and build some of these things
- 39:42from scratch on stream or just a video
- 39:45uploaded to kind of show, I guess,
- 39:47essentially what I made and maybe
- 39:48recreate and sharpen my skills a little
- 39:50bit more. Um maybe. I've got a lot of
- 39:52stuff on my to-do list, so maybe maybe
- 39:54not. It depends on the demand. Anyway,
- 39:55I'm going to cut the video from here. If
- 39:57you listened all the way through or to
- 39:59portions, then thank you very much. I
- 40:01hope it was interesting and enlightening
- 40:03and whatever else. I'll catch you
- 40:05around.
Try a question
Summarize this video
This video is a personal reflection from an engineer who was laid off by Atlassian, covering the technical work, the challenges, and the lessons learned during their time at the company. Key threads:
- Building an Open Service Broker (OSB) and its architecture , using Envoy as the proxy technology , and designing an XDS control plane .
- Deploying on AWS infrastructure , creating a machine image (AMI) , and a 24-month project recap .
- Extending a load-balancing platform, exploring Envoy extensions , and addressing edge compute versus centralized logic .
- The non-technical side: diplomacy and conflict resolution , mentoring , and navigating personality conflicts .
The speaker closes by expressing pride in the work and thinking about future projects to sharpen his skills . The video blends technical depth with a candid look at career transitions and long-term software maintenance .
Answers were generated by Viqu and pinned here so the demo is instant. Timestamps are clickable.
Pricing
Start free. Scale when it pays off.
Monthly subscriptions for regular use, or one-time credit packs when you need a top-up.
Subscriptions
Recurring credits every month. Upgrade or downgrade anytime.
Credit packs
One-time top-ups. Credits never expire and stack with your plan.
Questions