Introduction

How can you ensure your survey will guide you in making informed decisions?

Without proper design, survey efforts can be wasted, leading to meaningless data. If the data we collect doesn’t accurately reflect reality, we won’t be able to gain meaningful insights.

By defining what high performance looks like before designing our questions, we can gather meaningful insights for our department.

A well-designed survey provides very high quality data and will enable department directors and secondary leadership to gain situational awareness.

Key Principles for Designing Effective Surveys and Avoiding Pitfalls

To provide high-quality data, a survey must:

• Avoid biases while providing clear, concise questions and using effective data collection tools.

• Use as few questions as possible while maximizing information. Correlation analysis helps identify what’s necessary.

• Be based on a solid design that enables leadership to make informed decisions.

Common survey pitfalls include:

• Overloading with unnecessary questions.

• Lacking a strong conceptual basis, leading to irrelevant data.

• Collecting data but relying only on averages, missing deeper insights.

The best results come from choosing the right survey framework, followed by clear question formulation and proper scale usage.

The Telephone Game

Hey—It’s Denis.

You may wonder ‘Why invest so much effort into designing a survey? It’s just a survey!’ Afterall, you’re in the role of Director-equivalent because you make decisions, not to survey staff.

Your decisions are a result of your situational awareness and understanding. Most of those you will receive from business P&L and conversations with your reports. It’s a devious game of telephone.

In absence of valuable insights and awareness provided by quality surveys you will have to resort to the most frustrating way of making improvements: guessing.

Surveys guarantee to short-circuit that game. But to provide awareness, the surveys also have to be designed for purpose. You cannot just throw a random one in your teams’s inboxes.

Also keep in mind that spamming out low quality surveys that show no outcomes of improvements fatigues your department. Have you ever wondered—what answers does your staff provide when they are demotivated to fill out surveys calmly?

Surveys help you build your confidence in the quality of decisions you make. Lacking either of these two will invite feelings of imposter syndrome and inability to influence change in the organization.

Hey—It’s Patricia again.

At Flix, where I coach teams and individuals, my goal is to improve efficiency and reduce friction in our processes. When planning interventions at the team or department level, it's essential to have enough data to identify the real issues. This prevents us from focusing on less critical problems.

When someone refers to "the biggest problem in our department," I always ask, "How do you know this is the biggest problem?". To support answering this question, we’ve designed a survey and moved from basic analysis to more advanced statistical methods, allowing us to make better data-driven decisions.

Some key insights from the analysis include:

• Identifying underperforming teams.

• Pinpointing areas in the department that need attention.

• Analyzing consistency across teams.

• Finding groups where similar improvements can boost performance.

Read 1st analysis results

The entire process, from data collection to results, takes time, and the insights are only valuable if the right questions are asked. To ensure this, we focus on designing the survey correctly from the start.

Frame the survey first

The first step is selecting a framework to guide your question design. You can either use an existing framework or build your own, depending on your resources.

Using an existing framework

If your organization lacks the resources to create a custom design, you can use an existing framework. One option is the DORA Capabilities, which offers a comprehensive catalog of 18 topics. Questions can be crafted around each capability to suit your needs.

Building your own questionnaire from scratch

Alternatively, you can define your own principles of what makes a team high-performing. Many frameworks focus on three pillars, where a high-performing team:

• Does the right thing.

• Does it right.

• Does it fast.

From there, you define what each pillar means and develop principles that guide your question creation. For example, if you create five principles, three to four questions per principle will result in a total of about 20 questions.

Although time-consuming, aligning leadership on these definitions ensures you’re ready to start collecting meaningful data by the end of the process.

Conclusion

Crafting a well-designed questionnaire is essential for uncovering valuable insights into department performance. By choosing or creating the right framework, you can ensure your questions are comprehensive yet focused.

Follow the part 2 to learn more about how to optimize your questionnaire for better answer rate.