My Vision for Our Work Together

(This is a living document and will change and evolve over time. I’m writing this for my new team so that we may develop a vision together.)

This article outlines what I want us to accomplish together, and how I think we can get there. I want us to have a tremendous impact on the organization. I want us to lead by example, both as a team, and as individuals. I want our team to be seen as the very best. And I want us to get there by being our truest and most authentic selves. I want us to enjoy the journey together and to believe in what we’re doing.

On moving into the future: Ours is an older tech company with a long history. I remember us in the 90s. I never want us to feel burdened by where we came from. I never want us to do things because it’s the way that they’ve always been done. I want us to incorporate all of our learnings and experience to find a new and better way for tomorrow while meeting the demands of today. I want us to learn from all of our past mistakes, and to not make them again. I want to look at modern approaches, tools, and technologies, and find the way that makes sense for us as humans to adopt, borrow, and be inspired with them while understanding that technology and practices are tools and that there is no best tool to use, but only the best tools for the problems AND for the people solving the problems. We should always consider the people and the solutions together, knowing that we have to support and build tomorrow on what we create today.

On relationships: I’ve been through a lot. I know we all have. Life has been hard. Very often we show a shiny veneer of strength and stability. Our inner worlds can be much more tumultuous. I want us to connect as humans, not as co-workers and employees. I want you to feel safe knowing you will have some bad days. I want to give you my trust, knowing that you want to do your best and I want your trust knowing that I will do my best. I want you to feel protected when something goes wrong or when you make a mistake. I want you know that you can be honest with me and that I’ll do my best to protect you in those challenging moments.

On psychological safety: I commit to making you feel safe, protected, understood, healthy and cared-for. I commit to doing this to the very best of my ability. I commit to working tirelessly to get us there.

On communication: sometimes we may feel that we say things because it’s the right thing to say in a group (this is called “Groupthink”). Even if it’s not what we really believe, we may feel that it’s easier to say what people want to hear. I do not want us to do that. I want us to identify “Groupthink” and to stop it in its tracks. I want to encourage dissent. I want you to have strong opinions and to voice them. But I want us to hold our strong opinions loosely, so that we may come together, hear each other and be heard. I want to find the right way forward with honest agreement. If we cannot change opinions to meet our own, we must find the truth in other people’s opinions until we truly believe that it is the way forward. Or we must find a solution that’s better than any of the opinions on the table that everyone feels good with. I commit to you to never demand we move in a direction. I commit to you that I will encourage your dissent, do my best to understand your opinion, and find a way forward where everyone feels heard, understood, and in agreement.

On reward: We are all unique. Intrinsic motivators differ for each of us. I vow to connect with you as a human individual and to understand what really drives you. For some of us it is learning and personal growth. For some it is acknowledgement and recognition. For some it is seeing the reality and impact of the work that we do. We are motivated by all of these in some degree in different ways and at different times. I vow to give you the opportunities that you desire. I will devote myself to helping you grow and shine and make a difference in the world around you. I never want to reward you for something different than what we really want as a team (“Rewarding X but Hoping for Y”.) And I promise to do my best to work with you to ensure you are getting you what you need to connect with the work that you do, to have fun and feel good about it.

On measurement: What we are measured on is what we deliver on. I commit to measure you to show your strength and contribution. I commit to working to ensure others see this as well. I’ve seen people contribute great value but barely be recognized for their work – I commit to showing your contributions and ensuring your work is celebrated.

This is my vision for us as a team. How we get there will be up to us to discover. It will be an iterative process, and we must communicate and revisit the vision and ensure our strategies align and grow to meet this.

Remote Work: A Manual for Humans

Why I’m writing This

Welcome fellow remote workers. This article exists because, unlike myself, many people in the wake of COVID-19 are not working remotely by choice, and the teams do not yet understand how different working remote is. Many companies have trouble when they intentionally set out to create a remote work environment – how much worse is the problem when it’s suddenly forced on people and teams without time to figure out how to make it work?

When I started remote work, after the initial honey-moon phase, I found myself one day lying on my couch unable to move, unable to work, unable to socialize. Friends would ask me to come out and I would reply: “I’m in a dark place. I can’t make it.” Within a couple months of remote work, I was debilitated by anxiety. The time I spent in that place was, thankfully, very brief, but I will never forget the process. Problems with focus and anxiety that I was able to compensate for in the 9-5 office-day spiralled out of control. After I emerged from the depths of my own personal hell and started building better habits, I learned how to connect with other people and very quickly discovered that the seemingly-strong people around me silently struggled with the very same difficulties that I did. 

We started discussing mental health at work more, and I noticed a certain portion of new employees cyclically seeking medical intervention for focus and attention. I saw and heard people battle with their anxiety, depression, and focus over and over. With remote work, a snowballing guilt that stems from these pathologies compounds the issue, causing immense distress to the individual. Depression and anxiety may be precipitated by temporary conditions, but will become permanent fixtures of a person’s behaviour if not addressed quickly. And once they are a part of you, it takes a lot of work to dig out of.

I hope you can see here that the inner-realities of remote work can look different than many people realize or understand. The “digital nomad” lifestyle is seen as glamorous on instagram, but I promise you that it’s far less delightful in reality. If you are here, you’re likely getting a sense of this now. Yes, you have the flexibility to work where you want and when you want. But there is a dark side hidden under the shiny veneer of infinite flexibility. Don’t fret – there is a path forward.

I’m writing this because I do this all day every day, and my peers do this all day every day. I’m a unique tech worker – I’m a business grad and have worked in leadership so I have a lot of soft skills beside my technical chops. These days I find that I am very interested in the human problems in remote work and I discuss them with everyone around me. Becoming an effective remote worker is a process I strive to master so that I can help people overcome their struggles. I don’t have all the answers. Nobody does. But I can share my experience and what I’m doing to find my stride. As I have emerged from my own pain, I have learned how to connect with other people with my stories. I’ve gathered a set of mental models for remote work which I’m able to share with you here.

To summarize, I’m writing this as a guide to understanding why remote work is different and how to handle those differences, day by day, one day at a time. Let’s get into it!

Elastic NV, A Distributed Company

To share a little bit more about where I’m coming from, I want to share the story of the company that I was working for when I wrote this (I’m no longer there, but I believe there are some great examples to share). Elastic was formed as a distributed company, especially around the open source software “Elasticsearch.” Elasticsearch powers many of the products, services and companies that you interact with. I say Elastic is a “Household name” to people in technology, but to the end user the technology is hidden under the surface. If you go to Wikipedia and search for something, Elasticsearch powers that search. When you are matched on Tinder, when you find an Uber or Lyft driver, when you look for restaurant reviews on Yelp, when you search for a movie on Netflix, Elasticsearch is behind it. Much of the data analysis on COVID-19 is happening around the Elastic offerings as well. But the capabilities have extended so far beyond search. We’ll leave it there for now.

This company, Elastic NV, was formed as a distributed company. The qualities of the distributed company are used as a competitive advantage. For example, 24 hour support isn’t difficult as we have engineers living in whatever time zone you are in. 

As Elastic was born as a distributed company, the qualities of distributed work were evident through every decision leadership and HR made, through every thought, word, and vision statement written. Distributed work is in our DNA and documented in what we call our “source code” https://www.elastic.co/about/our-source-code and I will demonstrate what the company embodies throughout his article. 

9-5 Ethos Considered Harmful

The first thing to realize about distributed work is that trying to uphold an idea of a workday is fundamentally different. You have to reframe your relationship to work, you have to trust that people are doing their best, and not feel a need to correct them if they are having a bad day. You have to give yourselves the space and time to be “okay” and to encourage each other to do the same. The idea of “work-life” balance is fundamentally different when your work and your life don’t have a solid space/time boundary.

The first point in Elastic’s Source Code (“Home, Dinner”) highlights this point: 

There is no such thing as work-life balance. We are successful if we find balance in life. Elastic empowers you with the flexibility to do so. Be home for dinner, go for a run midday, care for a sick child, or visit a parent. Finding balance means being more innovative and efficient at work. Which makes for a better Elastic.”

Moreover, people will often have challenges in their “workspace” that requires this flexibility to thrive. Perhaps you have an autistic child that needs unique care as they develop. Or maybe your parents are ill – tending to them is a constant pressing concern. Maybe you need to get to your therapist so that you can talk about how to be your best self at work. Needing to be “seen” during core hours while a person is doing their best to manage their responsibilities in their home life is only causing them harm. If you are skipping therapy to work, core issues aren’t getting addressed. 

If people have meetings during the work-day, people will move their schedule to show up to them. We want to do our best when we are healthy and engaged. You want to take stress out of the picture as much as possible and it requires trust. Don’t look for the green light beside someone’s name as a signal of commitment. It’s offensive and harmful to do so.

If an individual is given the space and time to take care of their life, they will be less stressed and more able to find moments of engagement and “flow”. The result of understanding you have flexibility to manage your life is more output in the long run. We need to be aware of this quality of remote work. If someone needs to work a late night to get some work done that they need to deliver, they will do it if they are engaged. We must innovate to find the time and space where we are most effective. That requires taking care of ourselves and our responsibilities to allow ourselves to be in a good space and find the space for ourselves when we work best. It takes some experimenting and it takes making some mistakes.

My advice is to give infinite flexibility, encourage people to experiment, allow people the space to do what works best for them. This means changing our idea of the 9-5 work week and categorizing micro-management as strictly harmful. I’ll say that it’s positive to try to adhere to a set of core hours, and reasonable to ask people to be available during those core hours, but I wouldn’t be too strict about requiring someone to be sitting at their desk. 

Let me further highlight how this plays out: If you feel that you need to work a 9-5 work week, but are heavily interrupted and distracted during that time, you’re going to feel guilty and unproductive. The result of that is to work longer hours to meet responsibilities. Suddenly the online 9-5 day is upheld, but guilt-fueled nights and weekends become the norm to “catch up.” The problem is that stress goes up and engagement goes down, and those extra hours of work become less and less productive. This cycle continues until a person is on-line for 60 or 70 hours a week while putting out 20 hours of real work. It’s a pattern that appears over and over. This doesn’t work.

Again, core hours are good for communication, but you have to allow some flexibility when workers are in their own space. Abandon the idea now of everyone logging 40 hour workweeks inside 9-5, offer yourself and each other the flexibility to innovate and find what works. Meet during the 9-5 but don’t assume everyone is doing the real work during that time. Output will draw the picture of productivity, and a person will be most productive if they are given the space to manage their lives without a strict requirement to be sitting at their laptop in core hours. It’s healthy to maintain core hours, but having family around can be disruptive to that in a variety of ways, and the goal is to keep stress low and inspiration high.

As I’m sure you can see, we should be discouraging each other from working excessively. If you see someone online on the weekend, talk to them. Remember people are going to stumble and fall and get stressed – if you see signals of someone working many hours, talk to them about it because they may be in that guilt/shame spiral trying to “catch up” in an unmanageable manner. One of the “model managers” I encountered in my career, Erick E, would show up at the office on Saturday night, find me working there in the dark alone, and would literally force me to close my laptop and walk me to my car, tell me to leave it alone and to go have some fun. We need to embody this for each other and find a healthy and sustainable work-life balance to ensure our continued engagement in work and happiness in life.

It’s a paradox. It’s highly counter intuitive. But we need to allow ourselves the space to innovate and find balance for ourselves. We need to encourage each other and share our own process with each other. We need to talk to each other about how to draw boundaries at home, talk about how to set up a work space, talk about our struggles together so we all know we are in it together. Give yourself the space to take care of yourself and to find what works for you.

We all push sometimes, and we all take a little extra for ourselves sometimes. The hours balance out. If you have a short day to take care of yourselves, it’s more likely you’ll find some hyper-inspired day where you work a vigorous 12 or 13 hours and just don’t want to stop. If you’re taking care of yourself, those days appear more often. If you’re a rattled mess, you’ll never find them.

For the micromanager types struggling with the idea of giving trust and freedom to their employees, let me explain how this works itself out over time: Remember how I told you I was crippled with anxiety shortly after I started working Remote? I promise you, the guilt and the pain that I felt in my worst days stung enough to spring me to action. Any additional stress or expectation on me in those moments could have driven me out of the company in an anxious panic, and I’m really starting to find my pace now. It’s fairly normal for someone’s good habits to fall apart when they start working from home as it’s a completely new territory and has its own learning curve. I hear it over and over. Let it happen a bit – it’s a process learning to find balance. Trust that it will correct itself. “Wu-wei” (non-action) is the heuristic the Toaists talk about – messing with it makes it worse. Allow it, be there for someone who is struggling, and don’t judge. Suggest ways that one can build better habits. Share your own story and what is working for you, don’t demand anyone to be perfect today as there really is a learning curve to remote work. If someone really isn’t able to deliver, it’ll become evident in their output and you can have discussions focused on results. Clock-watching likely isn’t going to help someone who is in the middle of a battle with the demons that have suddenly appeared.

If things don’t improve for an individual quickly, then the risk is that the behaviours will become a part of who they are. It will become a habit. So those moments when we are battling with our inner world are some of the most important and defining moments – being able to look at what’s happening guilt free and adjust is the path forward. People will have a tendency to want to distract themselves if they’re anxious so it’s important to give a lot of space around these battles, talk about them, express your own so people know they aren’t alone. These moments may be some of the most defining moments as remote workers. If we are supporting each other, we can emerge victorious. If we fail to address those root problems, we will become disengaged and unable to find joy and inspiration in our work.

Engagement as a Signal of Success

Elastic does a lot to measure employee engagement. Engagement is a sense of being connected, involved. It’s a sense of contributing to something real, and seeing that the work we do as an individual is affecting the world in some way. It’s a sense in believing in what we’re doing, what our organization is doing. The result of engagement is more quality time spent working as we feel interested in solving the problems in front of us.

Engagement is the fuel for inspiration, and moments of inspiration create good ideas and better work. With engagement, a task is not something followed step by step, but a problem that is solved. If a person is inspired, they will really look at everything surrounding a task, find the very best way to deliver, and find ways to improve things around the task. Without engagement, a person is trying to complete the task. With engagement a person is inspired to do the very best they can for a team as they look at the problem.

Feeling supported, and feeling connected to a team and leadership, will ensure people really “show up”. You’ve seen these moments in yourself, and you’ve seen it in your peers. Put everything in place for yourself and the people around you to make this the reality.

If a person feels depressed, anxious, disconnected, they will not feel that great drive to conquer tasks. You’ve seen these moments in yourself, and you’ve seen it in your peers. Weed out anything that causes a person to move in this direction.

Creating Remote-Positive Work Culture

The result of a healthy work environment is always going to be better output in the long run. Pressing people for short term gains is a short term strategy and it doesn’t scale. I hope that it’s clear at this point that:

  • Personal wellness will create engagement;
  • Stress and anxiety kill engagement. 

A work culture should stigmatize pathological behaviour (micromanagement, working long hours) and celebrate healthy ones (social engagement, personal wellness.) Moments where short-term pushes are needed will arise, and you’ll want people feeling fresh for them.

There are a few items that can be used to influence these behavioural changes:

Vision and Slogans

Trying to define exactly what it is that makes remote work function for you and your teams is a great place to start. Write down some starting points – that’s what the Elastic Source Code https://www.elastic.co/about/our-source-code was and it evolved into a piece of our DNA that every employee understands and tries to embody.

A guiding vision is required to affect this change in the work environment, and you can start the discussion to influence that change, wherever you are in an organization. Say it over and over and over – communication is cited as a major reason organizational change fails (see forbes article and study cited: https://www.forbes.com/sites/victorlipman/2016/02/08/why-does-organizational-change-usually-fail-new-study-provides-simple-answer/#1457f21a4bf8).

Saying something over and over is a good way to influence behaviour. Make it a part of who you are, and highlight it as the solution to every problem to which it applies. Your ideas will become a part of everyone’s vocabulary if it’s on point. If you work in leadership, this is ever more important. Under-communication in failed organizational change effort is often described in order of magnitudes (10x or 100x more is needed.) Slogans are helpful as well and whatever vision you have should contain pithy bits of wisdom easy to pass around. They point the direction and heuristics people should follow. 

One of my favourite slogans embedded in Elastic’s work culture is “Assume No Malice.” This slogan is instructing us to suspend our emotional response to any communication, and to assume that the individual seemingly confronting us is trying to do well. If someone’s message seems harsh, take a breath and look underneath for their motivation and intention. Communication in the remote environment has its own challenges – every Elastician knows the slogan “assume no malice.”

Communication in Remote Work

Remote communication can be difficult for a variety of reasons. Time zones can come into play. You may need to define what channels of communication are considered to be a discussion, and what channels decisions become canon. The other thing to consider is that many traditional (what I would call “synchronous”) meetings can be replaced by asynchronous communication – especially status meetings and project updates.

Because remote work offers different tools, feel free to get innovative with what you’re trying to do. If you’re doing something on a meeting because it’s the way it’s always been done, consider innovating. Because we have calls, messaging (eg slack), and documents at our disposal, we have tools to use. Strategy is the goal (keep everyone in the loop) but there are different tactics (meeting, update a doc daily, or post something in a slack channel.)

The communication channels themselves deserve attention and some formality. A message to a co-worker or telling someone in a meeting that they need to do something might not be considered a task or decision. You may want to take decisions made in message channels and move them elsewhere – possibly in a ticket (eg Github for tech) or an email. If it’s ephemeral (something that occurs at a point and time and disappears like a slack message,) move anything important to a more permanent location that’s easy to find and reference in the future. 

The lack of social cues can let the most innocent messages seem harshly critical or malicious if the reader misreads it. As mentioned, the slogan “assume no malice” is in every mind at Elastic and is a constant reminder to all of us to suspend our emotional response and have another look. Get on a call if there is any confusion or perceived emotion. A message is just a message; it’s tone is created by the reader’s mind mixed with their current emotional state and that can cause an unintended tone when the message is received as there aren’t many other cues.

The other big communication drag in an organization is synchronous meetings meaning a meeting where everyone has to be there on the call to receive and share information. Status updates in particular are good candidates for innovating on. You may still want periodic calls to sync, but smaller status meetings like daily stand-ups can often be replaced. If you have 8 people in a one hour conference call, that’s one entire work day burned. Keeping on task is important, but there are a few “creative solutions” to these problems. We’ve recently set up asynchronous stand ups in Slack for our team by creating a room and sending a daily reminder. My team is spread across the globe from PST (Pacific Coast/USA) to ACT (Australia) and we meet to have syncs weekly still, but share our work status daily through messages. Much of the information shared on the details of our work is a good candidate for asynchronous communication as it can be read and referenced as needed without requiring people to gather on a call. We can drop our updates and consume them at our leisure. It’s easy for everyone to see you have a blocker when it’s written in a dedicated space everyone reviews daily.

A Model for Self Care

I believe self-care is the most important input to an individual’s success. Keeping stress low means more periods of vigor and inspiration. It’s up to you to find your best moments and you’re going to find more of them if you’re physically, mentally and emotionally in a good state.

There are a few things that I do that help me feel good about what I’m going.

Daily Checkin

Every day I take 15 minutes to sit and check in. I do a few things here:

  • I review a list of good habits that I’m trying to make a part of my life.
  • I write 3 achievable tasks down to try to accomplish in the work day.
  • I try to leave a bit of open ended time to think about how I’m doing.

I have a list of habits (originally on paper, now in the app “Habits” that I track). I’ll touch this list every day in the morning. It helps remind me of my goals around self care and productivity. I’ll take the time to hit as many of these habits around the daily check-in as I can.

  • Meditate for 20 minutes a day. (I use the Insight Timer app to set an alarm and track my streaks. It gamifies meditation and has guided meditations as well.) A single meditation session reduces stress. But meditation practiced daily will transform you in ways you cannot imagine. 
  • I write a quick appreciation, sometimes for the nice things, and sometimes for the hard things. For example: “I’m grateful for the anxiety I’ve experienced as a remote worker because it allows me to help other people who are suffering today.” If this sounds fluffy, there is real science behind this.
  • Pour 1L of water into a bottle and leave it on my desk.
  • Put on an Upright Go posture trainer for a couple hours.
  • Exercise. (Usually that’s kickboxing. In the midst of COVID-19, that means 250 pushups every other day with situps in between.)
  • Start one pomodoro. (More on this later) 
  • Check for blockers I can help people with (as a software eng, this means review PRs for a while most days).

I also have some items to eliminate bad habits:

  • Learn while code is compiling. This is a positive way of wording “avoid social media and news while waiting.”

I refer to this model of checking in “habit stacking”. My recommendation is to make one core habit – the daily checkin – and from there you can decide other things you want to add to that and just keep building. If you’re starting out and want to try this, just make a check-in in your calendar and track it in the Habits app on your phone. Click the checkbox for your daily check-in and it will be an unshakable part of your routine in no time – just make sure you do it every day. From there, you can add things to the list: do one set of pushups; sit and meditate for 20 minutes; do a pomodoro; call your mom and tell her you love her. Your time spent becomes a conscious decision and your actions start to reflect your intention to be better.

Many people say that their good habits fall apart quickly once they’re working remote – this is my antidote to that, to very intentionally build and maintain a set of habits and ensure they are re-enforced often.

Once you’ve got in the groove building your habits, it’s nice to leave some open ended time to allow higher level thinking about your tasks, habits and patterns as well. As an example, I’ve found I can prefer to pick up easy tasks if I’m not super inspired, but often the hard items are the most valuable to tackle, so I think about how to adjust my behaviour to deliver on larger pieces more consistently. Just sitting and thinking and checking in often generates some thoughts about what’s problematic. I’ve learned that talking to my manager about things that I think are difficult is a great place to connect and find better ways to work.

Managing Distractions

You’re starting where you are, and that’s okay. If you’re heavily distracted, and especially if you’re having trouble focusing on a daunting/confusing/unclear task, there is a technique that can help you break through this problem called “Pomodoro Technique” (https://en.wikipedia.org/wiki/Pomodoro_Technique.) The idea is that you wind a clock for 25 minutes (preferably a physical/analog kitchen timer that clicks away but you can use a phone timer). While the “ticking” is there, it’s your signal to avoid all distraction and focus on the task. If you notice yourself wander, that’s okay, but as soon as you realize what you’re doing, get back on task. When the timer ends, wind it for 5 minutes and do all of the things that distracted you: reply to people; check your email; get some water; even post something on Twitter in this time. If you can put your “distracting” habits somewhere and carve out quality of time, you can start exactly where you are and make that work.

The biggest benefit of doing this is that large/unclear tasks seem less daunting if you’re only going to give it 25 minutes right now. Just go and poke at it, start to understand it. You don’t have to finish it – just play with it a bit and see what happens. Touching it will often give you the interest and make it seem a little less daunting. After running at it for 25 minutes I promise you that big scary unapproachable task will suddenly have a few smaller hunks that seem less daunting, some further avenues to look, and you’ll feel motivated to keep going. The issue is the anxiety around unclear tasks can block us from starting (procrastination) – this approach helps break the ice. I believe that procrastination is tied to anxiety (both cause and effect) – this approach can help you move beyond that initial anxiety.

Setting Boundaries

One of the greatest stressors for the remote worker is the lack of boundaries implicit in remote work. If you’re at home, your family can access you any time. This sounds like a benefit but it’s a curse when you’re trying to focus. For myself, as a software engineer, I recognize when I’m in “flow” that interruptions are irritating. That irritation is correlated to your body dumping stress hormones. 

What’s worse is that the cost of that interruption isn’t the time it takes you to respond – in the middle of writing code or thoughts, it’s tearing down a mental context – a house of cards that you built up that you now have to start to rebuild after the interruption. For complex tasks, this may mean that we take longer to do it. For other tasks, it may mean that we take less care and do lower quality of work. 

The cost of an interruption is both time taken/quality and stress. These statements are not anecdotal generalizations, they are empirically demonstrated in studies. https://www.ics.uci.edu/~gmark/chi08-mark.pdf 

The other issue is that the lines blur very quickly between when you’re working and when you’re not. If you don’t create a boundary for yourself, you’ll both be living life and working all the time. There are two things that help this:

Dedicate space: when you’re in that particular space, you’re working. When you’re not, you’re not. Sitting in the living room on the couch with your laptop for the workday is dangerous as you live your life there too. Set up a box on the kitchen table today and stand and work. When you’re done, put the box away. Or set up a small desk. If you have room, dedicate one room the “office space.” For computing projects (music, art, photography), try to use a different computer and keep the work laptop for work.

Dedicate time: when you’re working, you’re working. Don’t bleed work into play and play into work. 8 hours of good work is better than 12 hours of work laden with distraction and play. 

Every day you make little decisions that will become your behaviour so you need to start right now – create the space, dedicate the time, do good work in that space and time so that you can enjoy the time away from work.

Media and Stress

It’s time to understand the effect that news and media can have on your state of mind. We have data in studies that can help us model and understand the effect that news has on us (see: https://www.ncbi.nlm.nih.gov/pubmed/17926432) If you watch the news for 15 minutes and then go to work, that exposure to the news will increase your anxiety and negatively impact your mood. Without some buffer to relax after seeing the news, you carry that stress into your next tasks. You watched some pandemic news, now you’re working, but you don’t actually realize that you’re in a worse state of being and doing a worse job because of it. Remember, we’re trying to find moments of inspiration – avidly following climate change or pandemic news isn’t helping you do that.

One of my favourite pieces of culture surrounding chat rooms is the dolphin emoji (🐬.) Posts it into a room before leaving to say “so long and thanks for all the fish” (a Douglas Adams reference.) Vigorously unsubscribe from any and all channels of communication that aren’t helping you. Limit your news and media consumption. It’s really not helping you. Sharing that article about impending global collapse isn’t going to change the world today. Whatever you take on, do it with intent. If you have fiat, check in on the stock market. If you have stock and have made a decision to leave it in, then stop watching until after COVID-19 pandemic has some kind of economic resolution. Unsubscribe liberally.

Socialization in Distributed Teams

Humans are social creatures and we’re happiest when we feel connected to the people around us. One of the biggest challenges in remote work is getting that when you’re physically isolated from all the people you’re interacting with. 

Consider when you sit with someone at lunch. You’re eating, but you are also talking about your interests and experience, and learning about other people’s. You may run into a problem and remember that your peer has faced something similar. In the remote context, you may not have the opportunity to have those conversations which adversely affects your ability to lean on the team around you.

To fill in these gaps, you need to allow some extra time in meetings to just chat. Book one-on-ones with your team, just talk to them, get to know them. They’ll appreciate it – trust me. It makes you feel connected but also fills in the details on where you can help each other.

If you’re having trouble getting beyond surface-conversations, there is one technique I’ve learned that really seems to get people to open up: expressing vulnerability. In the context of remote work, be brave and tell people about any and every one of your challenges. If it feels right, tell people about your inner world, meditation, your habits, your struggles. Tell them about the dark times in the past. You have to be a little careful here – you don’t want to divulge inappropriately. What you’re looking for is the point at which people “move away” and that’s the point that you’ve reached the limit of where you’re going to get, of where people aren’t comfortable any longer. You can start small, express a little, and see if they reciprocate. If they open up a bit more and tell you about their own struggles, then you’re making progress. Divulge a little more and see that they don’t start to back away – before long you can put everything on the table, they’ll put everything on the table, and you’ll have their complete trust. Make no mistake – this is the fastest route to building true and deep human connection, even with your co-workers. It takes practice and bravery to nail this but you have the ability to throw someone who is suffering a rope, and make them feel much less alone. You’ll build a true ally. I’ve been trying to practice this approach both in my professional life as well as with friends and family and it can be transformative for everyone brave enough to venture there with you. Just watch for the point people back away very carefully as you can overstep their sense of safety very easily – it’s not a common experience for people to be so open but I believe it’s an important heuristic for building deep human connection.

Finally, there are social events in the remote context. These can come in a few ways. We have some “always on” (AOn) zoom channels that people can drop into and just hang out, but the motivation isn’t always there to show up. In contrast, what seems to work is to schedule social time around some idea or task. “Quarantini’s” (happy hour) seem to be a popular social hour technique today – just getting on a call with co-workers and having a drink and talking about whatever comes up. I’ve seen teams play bingo on calls – the goal isn’t necessarily to play lots of bingo, but rather, to get everyone together around some common task without any stress or pressure to say anything. Even the most introverted human will warm up and interact with the room. Elastic also has a “coffee” room where a bot pairs you with some random person in the organization who is in the room and sets up a time for coffee – just a 30 minute chat to meet people that you wouldn’t normally talk to. These are great exercises to get you connected to more people and having some fun social experiences.

Anxiety, Depression, Focus and Medical Intervention

Anxiety, Depression, and arguably ADD/ADHD end up becoming behavioural patterns and a piece of who we are beyond the space and time that they first emerge. It’s possible to adjust our behaviour and to lift us out of those pathologies with inner work (Meditation, MBSR, Therapy, CBT or whatever works for you). The problem is that the pathologies are often so debilitating by the time that we realize we need to do something about our behaviour that we’re not able to start that work. If you’re a team lead and your family is there and you’re working all the time, you may not be able to change your situation as fast as you need to, and you may not be able to change your inner world as fast as you need to. Sometimes these situations require medical intervention (medication and/or therapy).

It’s okay to accept medication if you need it, but I would advise that it should be seen as something to get you to therapy or to start changing your behaviour. That’s not going to hold up for everyone, it’s only my opinion so take it with a grain of salt. But if you’re taking stimulants, antidepressants, or anti-anxiety medication, try to view it as a short term intervention. For ADD, your habits of distraction can be rewritten. For anxiety and depression, your relation to external and internal stimuli can change. Don’t be complacent if the drugs make you feel better – you should still be trying to compensate in your behaviours and change your patterns of thought and behaviour. Antidepressants (eg SSRIs) increase BDNF (brain-derived neurotropic factor) after approximately 6 weeks, which spurs neuronal growth in your brain. That means you’re making new connections. You can think and act differently in the face of stressors and emotions.

People think of anti-depressants as a daily treatment but more research shows that new “breakthrough” treatments for depression that do not boost serotonin, such as the NMDA-receptor-antagonist ketamine, but have some similar effects to weeks of SSRI treatment (increased BDNF and neuronal growth). New research is showing that our old ideas about depression being due to “low serotonin” may be wrong or over-simplified – it’s being stuck in the same groove that I believe is the issue. I’m over-simplifying the science myself in my statements here (there are many factors such as stress induced inflammation and astrocytic cell function at play which science is only just starting to uncover.) In my view, the goal of drug intervention is to spur change in your brain and you work from where you are – I mention this as I would like you to think about any depression/anxiety/ADD treatment not as the fix, but the starting point for change. These drugs often create dependence and the cure may be worse than the illness, especially with long term anti-anxiety medications. I have a friend who stopped taking anti-anxiety medication and it took him a year to feel normal again. The withdrawal had other negative effects in his life so the cost of using these drugs could literally take years to “pay back” when you try to stop. 

My 2 cents anyway – take it with a grain of salt. But be cautious and understand there may be a cost for long term medicating of your symptoms when the underlying problems need inner work to correct.

Supplements

There are a few supplements I would highlight for further research. Those I mention here allow your body to recover readily from stressful periods, both long term, and also to use in response to difficult periods at work. This section is not exhaustive and I am making inexact/blanket statements – do your own research and discuss with your doctor.

Adaptogens such as: Ashwagandha (eg ksm-66); Cordyceps; and Panax Ginseng. These compounds improve resilience to stress. They reduce cortisol in response to stress and generally will improve your health. Many of the adaptogens increase free testosterone so don’t be surprised if you make big gains in your workouts. These are generally safe to take daily for long periods of time.

L-Tyrosine is an amino acid in many foods that you eat that, when taken on an empty stomach, will cross the blood brain barrier and allow your brain to replenish catecholamine stock (dopamine, epinephrine, norepinephrine.) If you don’t need it, your brain has a rate limiting mechanism that protects your brain from producing any extra stress hormones, but if you’re chronically stressed and feeling depleted, 1 or 2 grams of L-Tyrosine on an empty stomach can bring back the color in life. In animal studies it’s been shown that L-Tyrosine acutely improves resilience to stress – you could consider taking it when you have that incident requiring some long hours as it’ll help you from fading. Take it short-term only as it will affect your serotonin and norepinephrine production negatively!

Kava can be used for short periods for dealing with acute anxiety. It’s been used for thousands of years in humans. Kava is quite safe (there may be some hepatic effects though), effective for anxiety (as effective as some benzodiazepine medications) and considered non-addictive, but be cautious with any GABAergic compounds like Kava and Alcohol and talk to your doctor if you’re trying to manage anxiety with drugs or alcohol with any regularity.

Any of the above should be available at your local health-food store. I’ve researched them all thoroughly and believe they are safe and effective for daily short-term use (a month or two daily if needed). I’m not providing exact information here, only highlighting a few avenues. Check with your doctor first and note any potentially adverse interactions, especially if you happen to be taking MAO inhibitors for depression.

And, please, avoid excess alcohol consumption. It is wrecking your body’s ability to handle stress.

Conclusion

I hope that this has helped to illuminate the qualities of remote work and to give you some new language and ideas to talk to your peers about remote work. Stay mentally fit and the rest will follow. Talk about the problems, come up with solutions beside your peers, be kind to yourself and be kind to the people around you. Good luck on your journey in remote work!

The Case for Manual Testing in an Automated World

There are many ideas that contradict one another in software. In one case, people may say that the DRY Principal – “Don’t Repeat Yourself” – should be adhered to without compromise. In other threads, people draw a line on what context that principal should be applied, and push very strongly for duplication in tests, as well as not sharing code between services or even bounded contexts of a modular application. Testing culture, similarly, has a lot of people believing that everything should be automated and that manual testing is dead. Here we’ll discuss the opposing idea: that manual testing still has a place in code culture, and discuss how and why it does, and where it fits into automated testing practices.

Types of Testing

Let’s highlight a few different layers and types of testing – while there are many typs of tests, we’ll focus only on a few:

  • Unit Testing: Automated testing of small bits of code. Can also include automated tests within a module or service that test integration between modules and components in the code (eg a slightly higher level test, but still isolated from other services.) True unit testing does not have an external database or any reliance on external services. As such, if built optimally, they should be extremely fast to run and have the benefit of being up-to-date documentation of the code by coupling to the behaviour.
  • Integration Testing: Combines modules or services together to test subsets of a system. Doesn’t need to run against a full environment. Can still be inexpensive to run as it doesn’t require a full environment and can be run from a module produced to combine modules/services.
  • System Testing: Automated testing that spans multiple modules or services, checking that they behave as expected when hooked up. Generally integration testing is run from “outside” of any particular service against a production-like environment. The tests makes requests against the public or internal APIs of the application to ensure the system as a whole, or subsets of components, behaves as expected. They are slow to run and expensive to maintain relative to unit testing.
  • Manual Testing: Similar to Integration Testing, manual testing will test against the entirety of the system but differs in that it is executed by people using the UI (or APIs as needed) to validate the system and its behaviour.

Manual Versus Automated Testing

Manual testing is often executed by dedicated QA teams in older organizations. Many modern technology organizations do not have explicit QA teams but may have engineering teams that aid in automated testing (The Software Engineer In Test role, for example, which joins teams to improve tooling and processes around automated testing.) The trend is to replace manual testing approaches with automated approaches and teams to support those goals and efforts. The major heuristic is that manual execution is wasteful as it doesn’t provide value going forward, and so building automation of all tests is logically better as an artifact is built in testing and maintains that the behaviour works forevermore.

There is some danger with automated testing. Of course, we know that software changes, so there is an overhead there in maintenance of the tests, and if they aren’t covering anything relevant, and aren’t easy to maintain and break frequently, then the cost of a test may outweigh the value it provides in its life, especially if the test isn’t covering important behaviour. It’s worse if the tests take a very long time to run.

I would suggest, then, that there is manual testing required to discover issues with a change, and that unit testing should be preferred over system tests, while system tests should be reserved for ensuring behaviour of a system is correct, and tests added to the system test suite be chosen with clear intention.

Manual testing can be entirely replaced by automated testing. But if it’s done indiscriminately, the cost of running and maintaining the automated testing may become too high. Likewise, if all test scripts ever created need to executed by humans, the cost will be far too high as a system grows in complexity and scope. I would argue that manual testing should be considered a piece of the development cycle apart from system testing. A human is better at “experimenting” than a script and can find blindspots that automated system testing can. As bugs are discovered by a human, and then fixed by a human, their fixes can generally include the faster, cheaper, more maintainable unit tests, rather than building system tests for some particular condition.

Summary

System test isn’t intended to cover a module or service’s particular behaviour, but only the behaviour of the components that touch. System tests and integration tests may only be concerned with testing behaviours that a unit test can not cover, and unit tests should test the specifics of a module/service that would be extraneous to cover in system tests. Manual testing should be done as a part of the development lifecycle, and issues identified/fixed be turned into unit tests wherever possible, while system testing be reserved for testing that cannot be covered with unit tests, especially confirming that components of the system are talking to one other correctly.

 

How to Test

I was talking to my very good friend about testing today and wanted to throw together a list of the heuristics that I’ve “discovered” which I found, personally, very difficult nuggets to discover through my career. Dan North in his article, “Introducing BDD,” describes the story arch in his career which sounds very much like my own:

The deeper I got into TDD, the more I felt that my own journey had been less of a wax-on, wax-off process of gradual mastery than a series of blind alleys. I remember thinking “If only someone had told me that!” far more often than I thought “Wow, a door has opened.” I decided it must be possible to present TDD in a way that gets straight to the good stuff and avoids all the pitfalls.

Example Code

Before we begin, I’m going to use a mutable Stack implementation in Scala as an example of code that should be easy enough to read and understand, regardless of the languages you’re familiar with.

Please note that I did not use a pure functional implementation with the intention of the code being more easily read by people coming from an imperative background!

A stack is a “last-in-first-out” data-structure that lets you place an item on the top of the stack (“push”) or retrieve and remove the item on the top of the stack (“pull.”) There is generally an operation called “peek” as well which will retrieve the item on the top of the stack without removing the value from the stack.

Screen Shot 2019-02-06 at 5.04.28 PM

You can find an example of a Java Stack here if you’d like further implementation details but it’s not necessary for comprehending the content in this article.

Naming Tests is Describing Behaviour

I’ve seen a lot of code in different environments in my career. One of the patterns I’ve seen in less experienced development teams is code like the Java JUnit example below.

public class StackTests {
    @Test
    public void testPeek() { ... }
}

The test name seems reasonable at first glance – it’s clear that the stack is the class under test by the test suite name, we’re trying to test the Stack’s “peek” method in the peekTest method.

Well, the problem is that we know what noun we’re testing, but we don’t know which specific behaviour is being exercised. Is it when it’s empty? Is it when it’s loaded with data? Should it do something different in these two cases?

One of the qualities of good tests is that they act as documentation to someone trying to read and understand the code. By describing the behaviour we are trying to test, we are also creating documentation that is forced to stay up to date with the code.

A good heuristic is to start a test name with “it should.” This forces the writer of the tests to focus on the behaviour instead of the noun. And this is the foundation of Behaviour Driven Development (BDD) – a focus on behaviour!

Let’s have another look at that JUnit test applying this practice.

public class StackTests {

    @Test
    public void itShouldReturnNothingWhenPeekingEmptyStack() { ... }
}

Most modern testing tools have been influenced by BDD so they will generally try to guide you toward describing behaviour. You can see in our JUnit example above that the test name is very long. Modern tools will help you organize your code by allowing you to describe different scenarios, and then describe the behaviour. Below is an example from ScalaTest which gives several different semantic options for describing your tests.

class StackSpec extends FlatSpec with Matchers {
  "An empty Stack" should "return None when peeking" in { ... }
  
  it should "return None when popping" in { ... }
}

How to TDD

Knowing the word TDD and understanding it in practice are two different things. TDD does not simply mean writing tests with your code, nor does it mean writing all of the tests before the code is written, rather, it is a practice of allowing the writing of code to be guided by the addition of tests, one at a time. The best way to highlight this is to demonstrate the addition of tests and code for our stack implementation.

The heuristic is: write the most general test that you can, and do the minimal work to implement the behaviours described so far in a test. This might mean hard coding a return value or stubbing features that aren’t yet under test as you go.

For example, let’s say we start with the test described above again:

class StackSpec extends FlatSpec with Matchers {
  "A an empty Stack" should "return None when peeking" in { ... }
}

The simplest thing to do here is to just hardcode the return value.

class Stack {
  def peek = { None }
}

Now, this obviously isn’t correct, but we chose the most general test, and then added the behaviour, and the test now passes. We can now add another test, and write the code to make it pass.

class StackSpec extends FlatSpec with Matchers {
  "An empty Stack" should "return None when peeking" in {
    val stack = new Stack(List())
    assert(stack.peek == None)
  }
  "A non-empty Stack" should "peek last value" in {  
    val stack = new Stack(List(1, 2, 3))
    stack.peek should equal(Some(3)) 
  }
}
class Stack(initialData: List[Int] = List.empty[Int]) {
  private var data: List[Int] = initialData.reverse
  def peek = { 
    if(data.nonEmpty) Some(data.head) else None 
  }
}

You can continue to add behaviour in this manner, and just continue to change the code to make the tests pass. Adding our first push behaviour, for example:

class StackSpec extends FlatSpec with Matchers {
  "An empty Stack" should "return None when peeking" in { ... }
  "A non-empty Stack" should "peek last value" in { ... }

  it should "push a value to the front of the queue" in { 
    val stack = new Stack(List(1, 2, 3))
    stack.push(4)
    stack.peek should equal(Some(4))
  }
}
class Stack(initialData: List[Int] = List.empty[Int]) {
  var data: List[Int] = initialData.reverse
  def peek(): Option[Int] = { 
    if(data.nonEmpty) Some(data.head) else return None 
  }
  def push(newValue: Int) {
    data = newValue :: data
  }
}

And so on, like this. There are two scenarios where TDD is very good:

  • finding and fixing bugs: add the tests to the behaviour to show its broken and to cover the expected behaviour (because there is obviously no test for it!) Then you have a repeatable mechanism for demonstrating the bug so you can easily do analysis to find the root cause.
  • building new features: this is really where TDD shines – adding new features is a joy with TDD.

Ping Pong – TDD With Other People

This is also a very fun thing to do with another person! The game of “Ping Pong” translates to TDD very well. With yourself and another person, you separate roles of test writer and implementor and you both take turns in a round of writing the behaviour and then writing the next test. So the activities above would be done like so:

  1. Person A:
    1. Writes the test: “An empty Stack” should “return None when peeking”
  2. Person B:
    1. Implements the minimum code to make the test suite pass.
    2. Writes the test: “A non-empty Stack” should “peek last value”
  3. Person A:
    1. Implements the minimum code to make the test suite pass.
    2. Writes the test: “A non-empty Stack” should “push a value to the front of the queue” 

You continue like this until the feature is done. It’s really a lot of fun if you both understand the “rules.”

  • When writing tests: Write the most general test to the most specific.
  •  When writing code: Implement the smallest amount of code to get the tests passing.

It’s not screw your neighbour per se… But if you can find a “creative” way to make the tests pass by writing less code on your turn… the option is there!

At some point a test that is introduced may force the hand of the pair to write a lot of code before the next test – this is okay and you can still pair as you normally would through that period of pairing, trading off the driver/navigator roles as desired.

Test Bottom Up: From Unit to Integration

I’m starting a role at Elastic and their Developer Constitution sounds like it could come from my own mouth. I deeply resonated with this article when I bumped into it, and the heuristic laid out here come from sometimes painful experiences so avoid those errors and follow the guiding principals. You’ll still get to learn from mistakes, you just don’t have to make them yourself.

This section on testing is a good heuristic.

Test bottom up. If you write code, write unit tests first. Write many of them. Write code so you can write many of them. Integration testing is the last step. Focus on adding more tests that execute fast and are easy to debug, like unit tests. This is crucial for developer velocity.

So the idea is to Unit test class/modules in isolation. And then to test them integrated together, and integrated with whatever dependencies it has.

 

 

 

ExUnit: Testing Named Processes

Named processes can be a bit of a pain to test if you need to setup data before they start.

If you try to start a named process in test, you will get an :already_started error.

There is a great pattern here that @jowensboggs from the Elixir slack channel tipped me off on.

The example here is when trying to merge commands from two timelines into a unified view, we want to test if any commands are due from one of the queues.

You can define the start_link method in your gen_server so that it inspects the args passed, and tries to extract the name, or falls back to __MODULE__.

 def start_link(opts) do
{name, opts} = Keyword.pop(opts, :name, __MODULE__)
init_args = opts
GenServer.start_link(__MODULE__, init_args, name: name)
end

then in your test you pass the name as a genserver argument:

 {:ok, pid} = start_supervised({ShiftSchedule.Worker, name: :my_test})

Finally, in your public API, you would need to optionally accept a pid:

 def commands_in_range(%DateTime{} = from, %DateTime{} = to, pid \\ ShiftSchedule.Worker) do
GenServer.call(pid, {:get_commands_in_range, from, to})
end

This lets you use __MODULE__ in the rest of your code base, while setting up any data needed for the process before starting it in your tests. Before this I was killing the processes so that the supervisor would restart it which is not good!

Real World Elixir Umbrella Projects

 

kisspng-umbrella-blue-royalty-free-illustration-blue-umbrella-5a89526f802b58.989132451518948975525.png

This article is a quick collection of my notes and references for working with Elixir umbrella projects in the day to day. It first covers my usual workflow and tools I use, and then moves into a more thorough discussion on the concepts and heuristics that I use as guiding principals around decisions of what to place in an umbrella app, where to break things down, and how to interact with umbrella apps. I briefly discuss querying data for presentation, although that topic deserves its own post.

Umbrella Projects

Elixir umbrella projects are an excellent option for building and growing applications over time. They are a good middle-ground between the monolith and micro-services based approaches, with less overhead than micro-services, but with the benefit of cohesion that micro-services enforce. Umbrella projects are a single project that contain multiple isolated apps (what might be called a module in other technology stacks). It can be difficult to understand exactly where the boundaries should be around apps in an umbrella project, and exactly how big or small they should be.

Creating a new umbrella project can be done with mix like so:

mix new my_project --umbrella

Inside that project, you can then create a new app with a supervised process. Simply move into the apps folder and ask mix to do it for you like so:

cd apps; mix new folder_name --module ModuleName --sup

That will give you a basic supervised process in a folder called folder_name in a module called ModuleName. Obviously change those to suit your needs!

There is a sketch of a module with hello world that can be discarded or added to, but further inside is an application.ex file that has the supervision and process description:

defmodule ModuleName.Application do
  # See https://hexdocs.pm/elixir/Application.html
  # for more information on OTP Applications
  @moduledoc false

  use Application

  def start(_type, _args) do
    # List all child processes to be supervised
    children = [
    # Starts a worker by calling: ModuleName.Worker.start_link(arg)
    # {ModuleName.Worker, arg},
    ]

  # See https://hexdocs.pm/elixir/Supervisor.html
  # for other strategies and supported options
  opts = [strategy: :one_for_one, name: ModuleName.Supervisor]
  Supervisor.start_link(children, opts)
  end
end

Now you can create a module with genserver behaviour and add it to the child list to get up and running. When you run the project, each of the application.ex files for each app in the umbrella project will be invoked, and the processes will be started and managed. They exist independently, yet can find and communicate with each other as needed through message passing. OTP and discussion of genserver and actor model implementations are outside of the scope of this discussion, and, if you’re here, I assume you have at least a basic understanding of the technology.

Working with Umbrella Apps

I’ll discuss this a little bit more shortly, but when I’m working in an umbrella app, I typically treat it entirely independently, running the tests inside that folder, and opening that as a separate project in my editor (emacs.) For me, re-opening the project means running a little snippet to produce a .projectile file in the root of each app‘s folder:

cd apps; find .-maxdepth 1 -type d -exec touch {}/.projectile \;

From there, I can open an app as a stand-alone project so I only see its contents. Alchemist will run the tests for only that app. I can then switch between the apps and their files quickly and easily using a blend of buffer management and projectile.

Granularity: When to Break Things into Separate Apps

When designing an application, you’ll commonly encounter the question of how big or how small to make each app in an Umbrella project. If you organized a system on post it notes describing all state-changes (events) that can occur, you would likely find that they can be grouped together around a few distinct topics. For example, an e-commerce site may have events such as ProductCreated, PriceChanged, ProductDescriptionChanged, as well as CustomerAccountCreated, CustomerInformationUpdated, and maybe ProductAddedToCard, CartEmptied. If we logically grouped these, we’d see that there are a few distinct entities that the events are related to. In Domain Driven Design terminology, these entities are generally referred to as aggregate roots. A little box can be drawn around these, and they can stand alone and independently without any coupling between them. In Elixir/OTP, it’s easy to see how we might send instances of these Commands and have them respond with the state changes (EmptyCart -> CartEmptied.) In general, these entities that our events exist around are the perfect place to draw a line and have in their own umbrella app. The little box we draw around these pieces, in DDD terminology, we refer to as the bounded context.

If you’re into Domain Driven Design, then you’ll have an intuition around what boundaries to draw around an aggregate root (a bounded context). If you have this level of thinking, then you likely know already exactly how big or small an umbrella app should be. I would suggest that, occasionally, it may make sense to have multiple bounded contexts within the same umbrella application if they are closely related, although it’s perfectly fine to make a rule that a bounded context always has its own umbrella application as well. In general, the actor model is an excellent fit for Domain Driven Design. If you’re struggling with where to break things down, I would strongly recommend investing time in reading Eric Evans’ or Vaughn Vernon’s works on the subject of DDD.

How to Work with Umbrella Apps and Bounded Contexts

Once you have some bounded contexts represented by different umbrella apps, you’ll likely not need to interact with many of them at the same time. Messages may be passed between them, so implementing a protocol between the contexts may require wiring from one to the other, but they will otherwise exist quite independently of one another. If they don’t, you may have demarcated at the wrong place. If you have the right granularity, then you should only have to put one bounded context in your head at a time.

Because these contexts exist so independently of one another, my preference for working on any one project is to treat it as an entirely unique project in my editor. For me, this means only having one project open at a time using projectile in emacs. For you that might mean opening each folder in the app as a separate project in sublime or VSCode but you can use the same approach.

 

What to Share Between Projects

Generally, we don’t share anything between projects, but I do like having a couple projects that are foundational and shared. No utilities or anything like that should be shared if at all possible. We don’t have any common util or shared project or anything like that. And if you want to be quite extreme about this, you may decide to not share any data between projects either, requiring that the aggregate root deal with any requests for data.

We have chosen to share db between the apps so that different apps can query a table through that module. We are not deploying micro-services in our use case at FunnelCloud, but if we were, we would insist on no shared data.

So we have a db project that’s used to interface with Postgres. We keep basic ecto models there and any then only very general queries. For example, to de-duplicate Kafka messages through restarts and deployments, we store the offset of the last processed message for the consumers within a bounded context. This offsets table is shared, so the model and queries are in this common Elixir module.

defmodule Db.Offset do
  use Ecto.Schema
  import Ecto.Query

  @primary_key {:id, :string, []}
  schema "offsets" do
    field(:value, :integer)
  end

def upsert(offset = %Db.Offset{}) do
  Db.Repo.insert(offset, on_conflict: :replace_all, conflict_target: :id)
end

def get(id) do
  qry =
    from(
      o in Db.Offset,
      where: o.id == ^id
    )

    Db.Repo.one(qry)
  end
end

We also have a protocol project that’s used to describe all commands and events. Because commands and events are shared between projects, it’s simpler to have the contract described and shared between them all, rather than sharing specific projects that might indirectly lead to inappropriate coupling.

We try to limit any other sharing between apps as much as possible. Otherwise, if you need to interact with a bounded context, you do so only through message passing.

What Kind of Messages Do We Send?

Working with the actor model in a purely functional language is a bit of a different paradigm, and, from a high level design perspective, it tends to look more like Object Oriented design than it does Functional Programming. Of course, in the details of the implementation, it is functional.

In Object-Oriented programming, the heuristic of good design is to bring data and behaviour together so that you tell objects what to do. Interacting with objects causes them to interact with other objects, and to change their state.

val car = new Honda.Prelude()
val person = new Person.Programmer()
person.getInCar(car)
person.lookAtOdometer(car) // 0 km/h
person.pushGasPedal(car)
sleep(1000)
person.lookAtOdometer(car) // 10 km/h

We never said “car.speed = 10.” We only told the objects what to do by issuing commands (getInCar, pushGasPedal). The objects do the the rest of the work by responding to those commands which can cause effects (state change). And the state changes that occurred could be described with events. The events, if they were emitted somewhere, might look like this:

class PersonGotInCar(person, car) extends Event
class CarAccelerated(car) extends Event

This approach of telling objects what to do by message passing is what Object Oriented programming was supposed to look like. Alan Kay, who coined the term Object Oriented, purportedly said:

“I invented the term object-oriented, and I can tell you that C++ wasn’t what I had in mind”. – Alan Kay, OOPSLA ’98

The underlying principal here is that we should tell objects what to do, not ask them about their state and changing it from outside of the object. Objects are not just data, they are the marriage of data and behaviour. The heuristic to remember is “Tell, Don’t Ask”.

TELL, DON’T ASK!

Now, functional programming looks different. In functional programming, there are no objects (save for multi-paradigm languages like Scala.) In functional programming paradigms, data and behaviour exists separately, such that functions act on data. State changes don’t occur, instead new instances of data are created by passing data into a function, and having data come out of the other side:

def push_gas(car), do: %{car | car.speed + 10}

old_car = %Car{speed: 0}
new_car = push_gas(old_car)
assert new_car.speed == 10

Purely functional applications are built by composing functions that accept and transform data without side effects or state changes. But real applications have state, and state changes over time. Enter processes/actors in Erlang/Elixir/OTP. Here, we can marry the two paradigms together because a process can hold onto some data, waiting to pass it to a function, along with a message, whenever a message is received, and then holding the output of that function and waiting for a message again. Messages can be passed to other processes that are also holding onto data, waiting to receive messages as well.

That heuristic of “Tell, Don’t Ask” that we discussed a few moments ago? It turns out that this is the heuristic that we want to use with our processes/actors in Elixir/OTP too. By sending Commands to processes, we allow a process to encapsulate state and behaviour and can build loosely coupled modules by adhering to this principal. It takes a little bit of getting used to but it’s a great way to build systems.

How to Read and Display Data?

(This is a difficult topic and there are many ways to handle this. I’ll quickly discuss my thinking, but there is no universal truth, only what works for you and your team with your knowledge and experience.)

If we need to be aware of the data inside of other processes for any reason, such as presentation, we can either choose to listen to events emitted from those processes (eg using something like Kafka to produce a queue) or we can otherwise have a read model somewhere that we can read from. For example, we could write the current state somewhere on every state change. Or we could directly query the process if we absolutely must. But separating the read concerns allows us to have a very succinct expression of the domain in the bounded context.

For a little more information on our approaches at FunnelCloud, we have event listeners set up in Rails to update the read model for pieces of the application (your usual event sourcing + CQRS architecture), and then the read model is displayed to the user. In other places, we’ve opted for a simpler approach where rails treats the data written by Elixir as read-only data for presentation to the user, while that same data is used as a recovery mechanism from the bounded context in Elixir. This approach is simpler than a pure event-sourcing implementation and works well for our use case there without the overhead of needing to maintain a journal of events. Both approaches are fine – while we do use event sourcing in some areas, real-world experience has made me a bit cautious in choosing where to use it as the journal needs to be maintained and migrated over time.