All articles

Tracking Engineering Work Alongside Support

We built Yetto because we believe that support teams work best when they can collaborate easily with engineering teams. When all the teams who focus on your customers can communicate with each other, your product will be better and your users will be happier.

Often, support organizations measure success with isolated metrics, such as the number of tickets closed (with no indication of what those tickets were) or the number of bugs filed (with no indication of how those might impact the user). How can you get your product teams to work directly with the feedback they get from users? One way is to provide access to the ticketing system that your support members are using, which often costs money per seat. You could have weekly stand-ups with Product, Engineering, and Support to review a curated list of tickets, which can be time-consuming and prone to bias.

If your engineering team already works with each other on GitHub, it makes sense to bring your support team there as well using Yetto. We'd like to demonstrate a few techniques that take advantage of GitHub's GraphQL API to measure your feature work against customer feedback.

Tracking internal issues versus those identified externally

Every user feedback issue created on GitHub via Yetto has the Yetto label applied to it automatically. But members of your organization can also create user feedback issues manually, as they always have.

It can be helpful to report on the ratio of internal to external issues as a means of identifying the strength of your test suite. If your users are identifying many more issues, it could suggest that you need more integration or feature test suites.

To implement this report, we'll lean on GitHub's search qualifiers, which lets us filter issues by label. If we prefix any label with a hyphen (-), we can also fetch issues that lack a specific label. Because we recommend that teams install Yetto on a repository that's separate from where the engineering and code work happens, our resulting GraphQL query would look something like this: 

{
  yetto_issues: search(
    type: ISSUE
    query: "repo:<YOUR_USERNAME>/<YOUR_SUPPORT_REPO> is:issue label:Yetto"
  ) {
    issueCount
  }
  internal_issues: search(
    type: ISSUE
    query: "repo:<YOUR_USERNAME>/<YOUR_CODE_REPO> is:issue -label:Yetto"
  ) {
    issueCount
  }
}

Note that the Search API considers issues and pull requests as one type, so you do need that additional is:issue qualifier.

The result of our query might look something like this:

{
  "data": {
    "yetto_issues": {
      "issueCount": 1
    },
    "internal_issues": {
      "issueCount": 53
    }
  }
}

We could get ✨ extra fancy ✨ and also check against the open and close states of the issues. That GraphQL query might look something like this: 

{
  yetto_open_issues: search(
    type: ISSUE
    query: "repo:<YOUR_USERNAME>/<YOUR_SUPPORT_REPO> is:issue is:open label:Yetto"
  ) {
    issueCount
  }
  yetto_closed_issues: search(
    type: ISSUE
    query: "repo:<YOUR_USERNAME>/<YOUR_SUPPORT_REPO> is:issue is:closed label:Yetto"
  ) {
    issueCount
  }
  internal_open_issues: search(
    type: ISSUE
    query: "repo:<YOUR_USERNAME>/<YOUR_CODE_REPO> is:issue is:open -label:Yetto"
  ) {
    issueCount
  }
  internal_closed_issues: search(
    type: ISSUE
    query: "repo:<YOUR_USERNAME>/<YOUR_CODE_REPO> is:issue is:closed -label:Yetto"
  ) {
    issueCount
  }
}

From these whole numbers, you can derive ratios by comparing the number of open issues with the number of closed ones; for example, (open issues count) / (open issues count + closed issues count). This can give an overall figure to how your team is doing.

Fetching Yetto issues closed by a pull request

GitHub has some nifty keywords that automatically close an issue associated with a pull request when the pull request is merged to the default branch. If your developers use this standardized verbiage when they open their pull requests, you can create a relationship between an issue and the code that addressed it.

Knowing this, you can use GitHub's GraphQL API to fetch every cross-referenced issue that was closed by a merged pull request. In addition, because every Yetto support ticket has a Yetto label, you can filter on just the issues that customers have written in about. That query might look something like this: 

{
  repository(owner: "<YOUR_USERNAME>", name: "<YOUR_REPO>") {
    issues(last: 10, states: [CLOSED], labels: ["Yetto"]) {
      nodes {
        title
        timelineItems(first: 1, itemTypes: [CROSS_REFERENCED_EVENT]) {
          nodes {
            ... on CrossReferencedEvent {
              pull_request: source {
                ... on PullRequest {
                  number
                  title
                }
              }
            }
          }
        }
      }
    }
  }
}

The response here might look something like this:

{
  "data": {
    "repository": {
      "issues": {
        "nodes": [
          {
            "title": "I have a problem...",
            "timelineItems": {
              "nodes": [
                {
                  "pull_request": {
                    "number": 52,
                    "title": "Fix the user's problem"
                  }
                }
              ]
            }
          }
        ]
      }
    }
  }
}

Here, title identifies the customer issue title, while pull_request shows the pull request that closed the issue.

Identifying which individuals have not responded to a ping

GitHub issues also allow you to @mention other GitHub users with access to your project to get their attention. With Yetto, this is a useful technique for a support member to quickly collaborate with someone else in the organization to address an issue, like an engineer or product manager.

However, sometimes notifications can be overwhelming, and the person a support team member notifies can easily forget to reply. Knowing the name of a GitHub user or team that was mentioned, you can use GitHub's GraphQL API to get a list of issues where they were mentioned, and the authors of the comments in those issues, like this: 

{
  repository(owner: "<YOUR_USERNAME>", name: "<YOUR_REPO>") {
    issues(
      last: 10
      states: [OPEN]
      labels: ["Yetto"]
      filterBy: { mentioned: "<ANOTHER_USERNAME>" }
    ) {
      nodes {
        comments(first: 10) {
          nodes {
            author {
              login
            }
          }
        }
      }
    }
  }
}

From this response, we can take a look at the author's GitHub username (represented by login) and note which issues lack a response from the mentioned user: 

{
  "data": {
    "repository": {
      "issues": {
        "nodes": [
          {
            "comments": {
              "nodes": [
                {
                  "author": {
                    "login": "Yetto"
                  }
                },
                {
                  "author": {
                    "login": "skalover95"
                  }
                }
              ]
            }
          }
        ]
      }
    }
  }
}

When you can parse this response, each object of the nodes array lists the commenters of the issue; you can then identify which of the authors are not ANOTHER_USERNAME.

What's next?

We're building Yetto because we believe that support teams deserve better tools. We want them to be communicating where the engineers are, not tossing issues over a wall and hoping that they're caught and addressed.

To see how Yetto works and sign up for the beta program to give it a spin, check it out on our home page. Or get in touch with us if you just want to know more!