Inspecting application state with the SOS debugging tools

In this post, we’ll cover how to use the SOS debugging tools to inspect variables from a process dump of a .NET Framework / .NET Core application.


Obtaining a memory dump

In this first example, we’ll use a running ASP.NET MVC 5 application hosted by IIS, but the steps here can be used on a normal .NET framework Windows application. Let’s start by taking a full memory dump of a running application.

Download ProcDump and copy it to the server that runs the application you want to debug. Obtain the process ID from the application you want to profile by using Task Manager, and then pass it as an argument to procdump.

procdump -ma <pid>

You should now have a dump named similar to w3wp_171229_151050.dmp in the working directory.


If you’re running several applications under a single app pool in IIS, it may be easier to debug by changing the app to run under its own application pool, which allows the ASP.NET app to run under a dedicated process.

Inspecting the ASP.NET application state (.NET Framework)

Now that we have a memory dump, it’s time to look at the suspended state of the application. Copy the dump file to your workstation, and then open it via File > Open Crash Dump in WinDBG. Your screen should look like this:

Load the SOS debugging extension, which will allow us to inspect the managed threads:

!loadby sos clr

Then, list the stack trace of every thread:



If get an exception when running this command and you are using IIS Express, try the command again. There appears to be a bug that throws an exception only for the first command run from WinDbg, which should not affect the rest of your debugging session.

You should see a lot of threads in the output. To narrow the results down, search for the namespace of your project in the output text.

We can see that there is an external web request being made in Thread 34. Let’s look at what external URL is being requested. Switch to the thread, and then run clrstack -p to get some more detailed information about each method call.

~34 s
!clrstack -p


You may see many arguments that contain the value <no data>. This can be caused by compiler optimizations; inspecting the state of these parameters is beyond the scope of this article.

The controller is present in this call stack, so let’s inspect the object instance by clicking on the this instance address, which is a shortcut for the !DumpObj command.

This instance contains a field named _request, which contained a field named requestUri, which has the original URI for this request:

That’s it! The commands vary slightly for dumping different field types.

.NET Core application on Linux


  • LLDB 3.9
  • Locally-built copy of the SOS plugin in the CoreCLR repo - instructions

In this next scenario, we’ll look at inspecting a core dump from a .NET Core app running on an Ubuntu x64 instance. The instance will have a core dump taken while a request is processing, which we will then inspect.

Take a core dump of the process using the createdump utility. These commands assume you have the coreclr repo checked out to ~/git/coreclr, and that you’re running an application built with .NET Core 2.0.

sudo ~/git/coreclr/bin/Product/Linux.x64.Debug/createdump -u (pid)

Load the dump in LLDB. This command also loads the SOS debugging extension.

lldb-3.9 dotnet -c /tmp/coredump.18842 -o "plugin load ~/git/coreclr/bin/Product/Linux.x64.Debug/"

After a few moments, a CLI will become available. Run eestack to dump the state of all CLR threads. If you get an empty output or a segmentation fault, verify that you are running the correct version of lldb and are loading libsosplugin from the bin directory, and that you have created the core dump with createdump.


There is an instance of HomeController in the stack of Thread 17. Switch to it to reveal more information about the current request. This time, we’ll inspect the state of an internal .NET Core request frame, since information about the current request isn’t as accessible as it was in ASP.NET MVC 5.

thread select 17
sos DumpStackObjects

Look for the address of Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Frame'1[[Microsoft.AspNetCore.Hosting.Internal.HostingApplication+Context, Microsoft.AspNetCore.Hosting]] in the output, and then dump the object. The name of this class might differ slightly based on the version of the framework you’re running.

Identify the QueryString field address:

Dumping that field reveals the query part of the URL the browser requested!

If debugging applications like this sounds interesting to you, join us! Thanks to Kyle Sletten, Justin Brooks, and Bradley Grainger for reviewing early drafts of this post.

Further reading:

Posted by Dustin Masters on January 09, 2018

Mitigating cross-site scripting with Content Security Policy

In this post, we’re going to look at using Content Security Policy (CSP) as a defense-in-depth technique to block script injection attacks.

When building website that hosts user-generated content, such as:

Great to be here!

It’s necessary to encode user-generated content so that browsers don’t mistake it for markup, and execute an untrusted script. This is easy to do for plain text, but what if a page needs to render user-generated HTML? Here’s an example of HTML that contains inline Javascript, which browsers could execute:

<p>Great to <b>be</b> here!</p>
<img src="" onerror="alert(0)" />
<a href="javascript:alert(0)">Hi</a>

This content must be sanitized before rendering. Libraries such as HTMLAgilityPack or DOMPurify provide a way to parse the HTML and strip out elements or attributes known to execute scripts.

Sanitization is important, but what if an attacker has discovered a way around the filter? This is where Content Security Policy comes in.

If the Content-Security-Policy header is present when retrieving the page, and contains a script-src definition, scripts will be blocked unless they match one of the sources specified in the policy. A policy might look something like:

script-src 'self'; object-src 'none'; base-uri 'none';

This policy disallows:

  • External scripts not hosted on the same domain as the current page.
  • Inline script elements, such as <script>
  • Evaluated Javascript, such as <img src="" onerror="alert(0)" />
  • Base elements, which could break scripts loaded from a relative path
  • Object elements, which can host interactive content, such as Flash

Whitelisting inline scripts

Sometimes it is necessary to run inline scripts on your page. In these cases, the nonce attribute on script elements can be used to whitelist scripts that you control.

<script nonce="00deadbeef">doSomething()</script>

A matching nonce must be present in the CSP for the script to run. For compatibility with older browsers, unsafe-inline allows scripts to run if the nonce tag is unsupported.

script-src 'self' 'nonce-00deadbeef' 'unsafe-inline'; object-src 'none'; base-uri 'none';

It is critical that this nonce is derived from a cryptographic random number generator so that an attacker can’t guess a future nonce. In .NET, RNGCryptoServiceProvider.GetBytes can be used to fill a 16 byte array:

using (var random = new RNGCryptoServiceProvider())
    byte[] nonce = new byte[16];
    return Convert.ToBase64String(nonce);

Whitelisting external scripts

strict-dynamic can be used to allow scripts hosted on a third-party domain to be loaded by scripts that you control. However, at the time of writing, this isn’t supported by all major browsers, so a host whitelist should be used as well as a fallback until it has broad support.

script-src 'self' 'nonce-00deadbeef' 'unsafe-inline' 'strict-dynamic'; object-src 'none'; base-uri 'none';

When using strict-dynamic, you will also need to add a nonce to any external scripts that are referenced.

<script nonce="00deadbeef" src="" />


Be careful what sources you whitelist. If any endpoints return JSONP and fail to sanitize the callback, it is possible to inject code. For example:

<script src="'';test"></script>

Might return window.location='';test({}) in the JSONP response, which would cause arbitrary code to be executed!

There are other policies that you can define to strengthen your site’s security, such as restricting where stylesheets are loaded from. This post only focuses on mitigating cross-site scripting attacks.

Further Reading

Thanks to Bradley Grainger and Kyle Sletten for reviewing this implementation.

Posted by Dustin Masters on December 21, 2017

‘in’ will make your code slower


The new in keyword (for parameters) in C# 7.2 promises to make code faster:

When you add the in modifier to pass an argument by reference, you declare your design intent is to pass arguments by reference to avoid unnecessary copying.

However, naïve use of this modifier will result in more copies (and slower code)!

This side-effect is implied by the MSDN documentation:

You can call any instance method that uses pass-by-value semantics. In those instances, a copy of the in parameter is created.

It’s also mentioned in passing in the readonly ref proposal:

After adding support for in parameters and ref redonly [sic] returns the problem of defensive copying will get worse since readonly variables will become more common.

Consider the example method from MSDN:

private static double CalculateDistance(in Point3D point1, in Point3D point2)
    double xDifference = point1.X - point2.X;
    double yDifference = point1.Y - point2.Y;
    double zDifference = point1.Z - point2.Z;

    return Math.Sqrt(xDifference * xDifference + yDifference * yDifference + zDifference * zDifference);

And assume this implementation of Point3D:

public struct Point3D
    public Point3D(double x, double y, double z)
        X = x;
        Y = y;
        Z = z;

    public double X { get; }
    public double Y { get; }
    public double Z { get; }

A number of C# features now combine in an unfortunate way:

  1. An in parameter is readonly
  2. Calling an instance method on a readonly struct makes a copy
    • Because the method might mutate this, a copy has to be made to ensure the readonly value isn’t modified
  3. Property accessors are instance methods

Every time a property on an in parameter is accessed in CalculateDistance, the compiler has to defensively create a temporary copy of the parameter. We’ve now gone from avoiding one copy per argument (at the call site) to three copies per argument (inside the method body)!

This is not a new problem; see Jon Skeet’s post on The Surprising Inefficiency of Readonly Fields. But using in makes it a much more common problem.


The solution is also in C# 7.2: readonly struct.

If we change public struct Point3D to public readonly struct Point3D (the implementation doesn’t have to change because all fields are already readonly), then the compiler knows it can elide the temporary copy inside the body of CalculateDistance. This makes the method faster than passing the structs by value.

Note that we could have achieved the same effect in C# 7.1 by passing the struct by ref. However, this allows the caller to mutate its fields (if it’s mutable) or reassign the entire variable to a new value. Using in expresses the intent that the caller will not modify the variable at all (and the compiler enforces that).


I’ve created a test harness that benchmarks the various combinations of in, ref, struct and readonly struct. (Note that I increased the struct size to 56 bytes to make the differences more obvious; smaller structs may not be impacted as much.) The full benchmark results are in that repo; the summary is:

Method Mean
PointByValue 25.09 ns
PointByRef 21.77 ns
PointByIn 34.59 ns
ReadOnlyPointByValue 25.29 ns
ReadOnlyPointByRef 21.78 ns
ReadOnlyPointByIn 21.79 ns


  • If you’re using in to express design intent (instead of ref), be aware that there may be a slight performance penalty when passing large structs.
  • If you’re using in to avoid copies and improve performance, only use it with readonly struct.

Posted by Bradley Grainger on December 07, 2017

MySQL Best Practices for .NET

We advise the following best practices for using MySQL Server with .NET code.

General Recommendations

Increase max_allowed_packet

The max_allowed_packet server variable controls the maximum size of a SQL statement that can be sent to the server, or the largest individual row that can be returned. Before MySQL 8.0, this defaulted to 4MiB, which is too small for many applications.

In MySQL 8.0, the new default is 64MiB; if you’re running an earlier server version you should set max_allowed_packet=64M (or higher).

Use MySqlConnector

MySqlConnector is a replacement ADO.NET connector library (with many contributions from Faithlife developers). It provides true async I/O, has better performance, and fixes many bugs in MySQL Connector/NET (aka MySql.Data).

Data Schema

Use COLLATE utf8mb4_bin

All text columns should use the utf8mb4 character set, as it allows the full range of Unicode to be stored. (See In MySQL, never use “utf8”. Use “utf8mb4”. for more details.)

By default, we prefer to use a binary collation:

  • It’s faster
  • It matches C# semantics: "a" != "A"
  • Many stored string values are IDs or tokens, which don’t need case-insensitive comparison

If you have human-readable textual data where you’re certain that you want the database to perform non-ordinal equality comparison and ordering, then consider using utf8mb4_0900_ai_ci (or decide if another collation is more appropriate for your use case).

Note: The default character set has changed from latin1 to utf8mb4 in MySQL 8.0.

Store bool as TINYINT(1)

In MySQL Server, BOOL is an alias for TINYINT(1). The MySQL ADO.NET connector understands this convention and will marshal TINYINT(1) back to managed code as the C# bool type (System.Boolean).

Use the BOOL alias when defining columns in your SQL statements. Do not use BIT(1) (which gets mapped as a ulong) to represent a Boolean value.

Avoid TINYINT(1)

As a corollary to the above, avoid explicitly using TINYINT(1). If you need a one-byte integer, use TINYINT (or TINYINT UNSIGNED). The (1) suffix simply indicates the “display width” (which is typically ignored by .NET programs), not the number of bytes used for storage.

Store Guid as CHAR(36)

To store a Guid, use CHAR(36) [NOT NULL] COLLATE ascii_general_ci.

The MySQL UUID() function returns a GUID in this format. The MySQL ADO.NET connector understands this convention and will marshal CHAR(36) back to managed code as the .NET Guid type.

The collation is ascii_general_ci because:

  • Using a one-byte-per-character character set allows MySQL to use fixed-length storage
  • Index prefix length limits are measured in bytes, not characters
  • Case-insensitive collation allows GUIDs to be queried using uppercase or lowercase hex digits

If you’re storing millions of GUIDs, you may wish to consider using BINARY(16) and performing custom data transfer. This will save 20 bytes per value, which may significantly reduce your data storage needs. Note that for a BINARY(16) column, MySqlDataReader.GetValue will return a byte[], but MySqlDataReader.GetGuid will reinterpret the value as a Guid.

Connection String Options


Uses Unicode when transferring queries to and results from the server, avoiding potential data loss from encoding errors.

Note: this option is utf8mb4 by default in MySqlConnector


With connection pooling on, but ConnectionReset=False, temporary tables and session variables from the last connection to the database will be retained. This is almost never what you want, and can lead to bugs.

Note: This option is True by default in MySqlConnector


If you’re OK with the risk of traffic between your web server and MySQL server being intercepted and read, then setting SslMode=None will use just a plaintext TCP connection. This increases connection speed (less handshaking to set up the connection) and reduces overhead (of TLS encryption) when transmitting queries & results. The overhead ranges from 10% (reading thousands of small rows) to 500% (reading a few large BLOBs).

Note also that SslMode=Preferred or SslMode=Required doesn’t really increase security because a MITM attack can just replace the server’s certificate; you need to specify SslMode=VerifyCA or SslMode=VerifyFull to ensure a secure connection.


Setting UseAffectedRows=True matches the default expectations of most ADO.NET programmers; for example, DbCommand.ExecuteNonQuery is documented in MSDN as returning “the number of rows affected”.

Note: This option is True by default in MySqlConnector

Posted by Bradley Grainger on October 30, 2017

Local Functions and Allocations

C# 7 introduces local functions. By default, the compiler generates very efficient code for local functions, even if the argument to the method is captured.

public int Function(int value)
    // Imagine this is a complicated implementation.
    // Note: this captures 'value' from the outer function.
    int Impl() => value + value;

    // perform argument validation (and early-out) here, then delegate to implementation
    return value == 0 ? 0 : Impl();

The decompiled C# compiler output is:

private struct <>c__DisplayClass0_0
    public int value;

public int Function(int value)
    LocalFunctions.<>c__DisplayClass0_0 <>c__DisplayClass0_;
    <>c__DisplayClass0_.value = value;
    if (<>c__DisplayClass0_.value != 0)
        return LocalFunctions.<Function>g__Impl0_0(ref <>c__DisplayClass0_);
    return 0;

internal static int <Function>g__Impl0_0(ref LocalFunctions.<>c__DisplayClass0_0 ptr)
    return ptr.value + ptr.value;

The struct incurs no allocations, and the JITter will inline the local function so it’s extremely efficient at runtime.

In order to hide the implementation in C# 6, you would have had to use a local Func (see decompiled code):

public int Function(int value)
    Func<int> Impl = () => value + value;

    return value == 0 ? 0 : Impl();

This would allocate an instance of a compiler-generated class and an instance of Func<int> whether or not Impl was actually invoked. The C# 7 code is much better. (Read more about local functions versus lambdas.)

However, there is an edge case in C# 7. If your local function would be implemented with a compiler-generated class (two examples I’ve found are iterator methods and async methods), then that class will always be allocated when the outer function is invoked, whether it’s used or not.

public IEnumerable<int> Function(int value)
    IEnumerable<int> Impl()
        yield return value;

    return value == 0 ? Enumerable.Empty<int>() : Impl();

The decompiled C# compiler output:

private sealed class <>c__DisplayClass0_0
    // iterator state machine elided

public IEnumerable<int> Function(int value)
    LocalFunctions.<>c__DisplayClass0_0 <>c__DisplayClass0_ = new LocalFunctions.<>c__DisplayClass0_0();
    <>c__DisplayClass0_.value = value;
    if (<>c__DisplayClass0_.value != 0)
        return <>c__DisplayClass0_.<Function>g__Impl0();
    return Enumerable.Empty<int>();

If the “early out” condition is taken >90% of the time, and if this function is called frequently, you may wish to avoid the overhead of the unnecessary allocations of the compiler-generated class. Fortunately, there is a workaround: instead of implicitly capturing the outer function’s parameter, explicitly alias it:

public IEnumerable<int> Function(int value)
    IEnumerable<int> Impl(int value_)
        yield return value_;

    return value == 0 ? Enumerable.Empty<int>() : Impl(value);

The decompiled C# compiler output:

private sealed class <<Function>g__Impl0_0>d : IEnumerable<int>, IEnumerable, IEnumerator<int>, IDisposable, IEnumerator
    // iterator state machine elided

public IEnumerable<int> Function(int value)
    if (value != 0)
        return LocalFunctions.<Function>g__Impl0_0(value);
    return Enumerable.Empty<int>();

[CompilerGenerated, IteratorStateMachine(typeof(LocalFunctions.<<Function>g__Impl0_0>d))]
internal static IEnumerable<int> <Function>g__Impl0_0(int value_)
    LocalFunctions.<<Function>g__Impl0_0>d expr_07 = new LocalFunctions.<<Function>g__Impl0_0>d(-2);
    expr_07.<>3__value_ = value_;
    return expr_07;

Now the “happy path” has zero allocations and the compiler-generated class is only instantiated if the local function is actually called.

I wouldn’t recommend doing this by default, but if profiling indicates that extra allocations are a problem, you may want to perform this refactoring to avoid them. I strongly recommend checking the compiled IL to ensure that you’ve solved it. If your outer function has many parameters or locals and (due to a typo) you inadvertently capture just one of them, then the optimization won’t kick in.

And note that in the typical “argument validation” use case for local functions, there’s no point in doing this because the outer function should call the local function 100% of the time. (The only reason it wouldn’t is if a boneheaded exception is thrown because the function was called incorrectly.) This refactoring is only useful if the local function ends up being called extremely infrequently.

I encountered a “real world” example of this issue in MySqlConnector. When I refactored the code to use local functions, I noticed an increase in allocations (and a decrease in performance). The solution was to avoid implicitly capturing local variables in the local functions.

In that specific scenario, the MySQL client library was reading bytes from a TCP socket. Most of the time, the bytes have already arrived and are in an OS buffer. Thus, we can immediately (and synchronously) return a ValueTask<int> containing the number of bytes copied into the caller’s buffer. Infrequently, we have to asynchronously wait on network I/O. Only then do we want the overhead of allocating the compiler-generated async state machine in order to return a (wrapped) Task<int> to the caller.

Posted by Bradley Grainger on August 02, 2017

Usage Guidelines for HttpClient

The HttpClient class is used in modern .NET applications to make HTTP requests. It was introduced in .NET 4.5 as a replacement for HttpWebRequest and WebClient.

This post collects some usage guidelines for HttpClient that may not be obvious.

Reuse HttpClient instances

Per MSDN, “HttpClient is intended to be instantiated once and re-used throughout the life of an application.” The rationale is mentioned in that MSDN article and described in detail in this blog post.

All HttpClient methods for making HTTP requests are thread-safe, so as long as you don’t change any of its properties (BaseAddress, DefaultRequestHeaders, Timeout, etc.) after you start making requests, reusing the same instance on any thread will be fine.

The simplest way to reuse HttpClient instances is to create a static readonly field in the class that needs to make HTTP requests. Of course, that still results in one instance per class, so consider allowing the HttpClient instance to be specified via constructor or method parameter.

HttpClient is disposable, but there’s no harm in never disposing it if it is used for the life of the application.

Example: The HttpClientService class of Facility.Core accepts an HttpClient instance in the constructor, but if it isn’t specified, it uses a static readonly instance that is never disposed.

Dispose HttpRequestMessage and HttpResponseMessage

The SendAsync method is the most flexible way to send an HTTP request with HttpClient. It accepts an HttpRequestMessage and returns an HttpResponseMessage. Note that both classes are disposable; be sure to dispose of instances of both classes when you are finished with them.

Example: FacilityCSharp has tests that run a number of HTTP requests in a row that POST a JSON body and process a JSON result. After a few dozen requests, HttpClient would asynchronously deadlock. I never got to the bottom of it, but I did find a solution: dispose the HTTP requests and responses.

Handle timeouts properly

When an HTTP request times out, an OperationCanceledException exception is thrown, not a TimeoutException.

The default timeout is 100 seconds. If you are using a single HttpClient instance (see above), you’ll want to make sure that HttpClient.Timeout is set as high as any request might need.

To use a shorter timeout for certain requests, use CancellationTokenSource.CancelAfter and send in the corresponding cancellation token. If you already have a cancellation token, use CancellationTokenSource.CreateLinkedTokenSource and call CancelAfter on that. If you need to distinguish between the two types of cancellation, check if the original cancellation token is cancelled.

Example: This StackOverflow answer demonstrates combining a cancellation token and a timeout.

Respect DNS changes

I haven’t attempted to reproduce this behavior, but apparently using a singleton HttpClient doesn’t respect DNS changes. For more information, read this article and this article.

Apparently you can use ServicePoint.ConnectionLeaseTimeout to work around this problem (see linked articles).

Aside: FacilityCSharp does not address this problem, but if you’re a Facility user and experience it, please file an issue and let us know!

Posted by Ed Ball on March 30, 2017

Getting Hired at Faithlife

Interviewing is hard. It is difficult to measure the whole of a person’s experience, expertise, and potential in such a short period of time. As hard as it is for me, I know that it is even harder for you. I do my best to make everyone comfortable, but it is hard to help people get past the nervousness. I think it will help if you are prepared. So, in an effort to help you do your best, I’ve prepared a few notes on getting hired at Faithlife.

Your Interviewers

First, a little bit about your interviewers. We are all engineers actively contributing code at Faithlife. We all volunteered to spend time with you and we all want you to do well. If there is something we can do to make you more comfortable during your interview, let us know.

How to Apply

The Minimum

Send us your GitHub, LinkedIn, StackOverflow, or a quick note. If it can be established that you meet the requirements from what you sent, you will hear from a recruiter.

Make an Impression

Make a good impression. It may be enough to get you a phone interview if you are a little short of the requirements on paper or carry you through a misstep in a subsequent interview.

Before you do anything else, do some research on Faithlife. Be able to answer this question: “Why do you want to work at Faithlife?” You will be asked this question in at least one of the interviews; likely all of them. You should be able to answer it. Write a cover letter (which answers the aforementioned question) and attach a resume. If you want to stand out when I evaluate your application:

  • Be interesting. A well written cover letter will make your application stand out. Write something that highlights your personality. Help me get to know you.
  • Contribute to one of the Faithlife open source projects
  • Contribute to someone else’s open source project
  • Do something else interesting. When I applied, I spent about an hour putting together a simple Bible search sample program using Lucene and C#. It was very basic, but demonstrated that I could write functioning code and knew something about Faithlife’s products.

Phone Interview

You got a response and your first interview has been scheduled. Great job! We are going to sit down for 30-45 minutes and chat. My goal will be to evaluate whether you have a basic understanding of data structures and proficiency with your preferred language. I will ask some questions and you will write some very basic code. FYI, when I ask you, “what is your preferred language?” which language you choose will not have any impact on your outcome. It will just tell me which questions I should ask. Choose the language you want to answer questions about. Your preparation should include:

  • Practice answering interview questions on data structures and your preferred language.
  • Practice writing code on one of the many practice sites. Pick your favorite, but here are a couple examples:
  • What is important to you about your working environment? The people you will be working with? The projects and products you are working on? When you have the opportunity, ask good questions.


We are experimenting with replacing a portion of the in-person coding with a homework project. If you are selected to participate, you will be assigned a small project to complete on your own time. Pay attention to style. Make sure it compiles and functions as intended. Take some notes. Be prepared for an in-person code review with questions focusing on decisions you made, implementation details, etc. If you have time, do the bonus work.

Panel Interview

Nailed it. Your panel interview has been scheduled. You will be meeting with 3 engineers for about an hour and a half. We will discuss your past work and you will write a lot more code. Remember, your interviewers want you to do well. We are all pulling for you. Your preparations should include:

  • Review any recent work that you’re proud of and be ready to discuss it in detail. Demonstrate your passion for these projects and what you learned from each.
  • More practice writing code on one of the aforementioned practice sites.
  • Practice testing your code. Pay attention to base and edge cases.
  • Practice talking through the code. Demonstrate that you understand what it’s doing and explain why. An incomplete solution that is well defined with a plan explained is usually better than a complete solution that is not articulated clearly.
  • What is important to you about your working environment? The people you will be working with? The projects and products you are working on? When you have the opportunity, ask good questions. Sound familiar? Don’t be afraid to ask the same questions. Get some different perspectives.

Pair-programming Interview

Congrats! The process is selective and you are among a very small number of candidates that make it to the pairing interview. You will be spending a day with us at either the Tempe or Bellingham office. What to expect:

  • When you arrive someone will show you around our campus and tell you about the different departments. Grab some coffee, or if you agree with me and think coffee is foul, grab a soda.
  • The interview portion will involve 2 pairing sessions, each lasting around 2 hours. You and your partner will be working on a problem, either real or from a set of interview pairing projects. We are evaluating your ability to communicate, collaborate, and ship working code. Your preparations should include:
    • Become familiar with one of the languages we use during this interview: C#, JavaScript, Objective-C, C++, or Java. You will be evaluated based on your experience, but building a basic understanding before you arrive will make you productive faster.
    • Before you get started, talk through the problem with your partner. Ask questions and build a plan together. We are evaluating your ability to communicate and collaborate as much as your ability to be productive.
    • Plan how you will test.
    • You may not finish the project. That is ok. Use the time you have to demonstrate your ability to solve problems with code productively and collaborate.
  • If you are with us on a Thursday, you will have an opportunity to attend “Demo day.” No need to prepare a demo. All of us (the engineers) and a few folks from the management and executive teams meet weekly to show off what we are working on and learning.
  • After demos, you will head out to lunch with a few engineers. This is a good time to ask questions about the culture at Faithlife and get to know some of your potential coworkers. Plan what you want to talk about, show passion and interest here.
  • Ask the good questions you prepared again.

Interviews with the Development Manager and the CEO

These may be held the same day you are on-site or at a later date, depending on availability. I haven’t performed any of these interviews, so I don’t have much advice. Ask good questions. Research the people who are interviewing you. Bob (our CEO) has written many helpful articles; like this one. The research will give you insight into how Faithlife runs and will likely inspire a few great questions. Good luck!

The Aftermath

If it went well, you have a start date! Congrats!

Connect with your interviewers and the other folks you met at Faithlife. Introduce yourself and ask for advice on preparing for your first day. If you have trouble finding those folks, connect with Leigh. Leigh is our Development Recruiter and would love to chat with you about your experience interviewing and answer any questions you have about working here. During your first few weeks at Faithlife, take some time to reach out and invite your interviewers to lunch. Ask them what you could have done better. Tell us how we can improve. This is the only time we can give constructive, personal feedback about interviews. Any constructive feedback you have in return is greatly appreciated. We all want to get better. Embrace openness (core value!) and help us get there.

If it wasn’t a great fit…

Interviewing is hard and we don’t want to take a chance on a bad placement. If we aren’t a confident “Yes!” the answer is “No.” We pass on some possibly great candidates because there just wasn’t enough evidence in the process to convince us. We value growth (core value!), so we are working on improving our hiring processes. If you have any feedback about your experience, we would love to hear it. These are the two most common reasons we have to pass on people and what to do about them:

  • Candidate hasn’t been pursuing mastery of their craft. They didn’t show evidence of enthusiasm for software engineering. If this is you:
    • Start reading. This is a good list to get you started.
    • Learn the ins and outs of the languages you are using day-to-day. You should have a firm grasp of how those technologies work, why they work that way, and where those technologies start to break down.
    • We move fast. Constant growth is an expectation. If you aren’t already actively pursuing knowledge and improving your technical skills, start.
  • Candidate doesn’t know their own weaknesses or limits. If this is you:
    • Growth (core value!) requires constant self-evaluation and the pursuit of actionable, critical feedback. If you aren’t aware of your shortcomings, you may not be doing either of these, start.

Keep pushing. Give it a few months or a year, then give it another shot. There isn’t any rule against re-applying. The next time you apply, tell us about all of the things you have done in the interim to improve. Recovering and learning from failure shows maturity which is an important part of being an effective engineer.


Remember, it is safe to assume goodwill. We want you to do your best. We know it is a difficult situation and we want to make it as comfortable as possible. Take your time, take a deep breath, and take us up on our offer of a beverage. If you need it, take a break. Best of luck!

Thank you Auresa Nyctea, Dave Dunkin, Leigh Vander Woude, Patrick Nausha, Todd White, and all of the other folks that helped put this together.

Posted by Jared Wood on February 03, 2017

Async and Await in WPF

In our last video, we looked at using the async and await keywords in a console application. This week’s video uses async and await in a WPF application.

First we create a simple event handler using async and await. Then we simulate what happens behind-the-scenes with await by implementing the same behavior using continuation tasks. Just like await, we capture the SynchronizationContext and use the Post method to run the continuation on the UI thread.

Next we use DBpedia’s SPARQL endpoint to asynchronously execute a query against its structured data from Wikipedia. We then see what happens when an exception is thrown in an awaited task.

Stephen Toub has an excellent three-part article (2) (3) on await, SynchronizationContext and console apps.

Posted by Scott Fleischman on June 03, 2013

TAP Using Tasks and Async/Await

With our two previous videos on Starting Asynchronous Work Using Tasks and Continuation Tasks, we are in an excellent position to use the new async and await keywords in C# 5.

This week’s video converts a simple synchronous method to async following the Task-based Asynchronous Pattern (TAP) using two different implementations.

  1. The first implementation uses Tasks, continuations and Task.Delay
  2. Then second uses the new async and await keywords, resulting in code that is very similar to the synchronous version. It also uses the Task.WhenAll method to asynchronously wait on multiple tasks.

For further reading, see:

Posted by Scott Fleischman on May 23, 2013

Continuation Tasks

Last week I posted a video on Starting Asynchronous Work Using Tasks. This week’s video is on Continuation Tasks. Continuation tasks allow you to control the flow of asynchronous operations. They are especially useful for passing data between asynchronous operations. Continuation tasks are normally created using the Task.ContinueWith method. They also can be created using methods like TaskFactory.ContinueWhenAll.

Posted by Scott Fleischman on May 17, 2013