Methodology @ Proposal Stage

How do you choose a methodology?

Not in isolation, and not last. Methodology follows from your topic and your research questions, and if either of those is poorly defined the methodology will not fit no matter how carefully it is chosen.

The chain runs: topic → research questions → theoretical framework → analytical approach → instrument. Choose the wrong framework and you end up with the wrong analysis, which means the wrong instrument, which means data that cannot answer your questions.

One practical warning: be wary of a Chair who wants a methodology commitment at proposal time without the reading to support it. The proposal binds you, and you carry the consequence.

What does methodology have to settle at proposal stage?

Finding Topic Relevant scholarship using your chosen methodology

If you know what methodology you will be using (e.g., the TAM, or the UTAUT), search Google Scholar for research in your topic's field that uses your methodology. This is a quick way to find highly relevant research for your review of literature and for discussion in your concluding chapter. Finding topic relevant scholarship can also help you to find your survey instrument as it will show you what research instruments scholars doing work in your field are using.

Searching scholarship to find what methodology is appropriate

• Are you comfortable with associated analytics?
  • Quantitative.
    • Quant based studies result in numeric values representing research answers. A Likert scale is the familiar one — strongly agree to strongly disagree, usually across five or seven points. You have filled in dozens of them. There are others as well!
  • If statistics sound a little intimidating do not worry. Statistical help is widely available and standard statistics packages are available for download at heavily discounted prices for students.
  • Statistical analysis is performed using software packages such as IBM's SPSS or others. Open-source packages are available e.g., JASP which provide similar functionality to SPSS and there is also extensive support for statistical analysis built into Microsoft Excel©.
  • We provide help with surveys, data analysis, and statistics and the need to perform statistics should not bar you from doing a quantitative study.
  • Setting up survey questions can be a challenge…and often? A big one. The right survey questions can result in easy analysis and good results. Survey questions that do not align with your topic, your framework and your analytic approach cause three problems. The analysis becomes difficult. The statistics come out poor. And you spend a great deal of time forcing results to fit research questions they were never going to answer.

Qualitative survey question setup

• Qualitative methodologies include grounded theory, discourse analysis, ethnography, and interpretative phenomenological analysis.
• Avoid Phenomenological approaches. There is no one common understanding of what phenomenology is. Thus, interpretation, in terms of how to conduct phenomenologically-based research is tricky and there is considerable disagreement on many issues related to such research. This kind of research asks what an experience was actually like for the people who lived it. It needs a researcher who can feel with a subject without being swayed by them, and who can hold a steady focus while the material arrives in very different forms — what people say, how they behave, what the setting shows which may include not only observations and interactions with subjects but also records or objects created by the subjects. In other words, you need to be friendly, empathetic but very strong willed in order to maintain topic focus in non-structured environments.
• Software packages such as NVivo exist to assist in qualitative analysis but have significant costs and learning curves. Worse? They have been subject to a good deal of criticism.

Why this is the worst possible place to trust an AI

Theoretical or conceptual framework — which do you need?

They are not the same thing, and many candidates are asked for one while being given guidance about the other.

A theoretical framework rests on somebody else's theory. You adopt an established theory — one that already exists, with a literature behind it — and your study operates within it. The theory tells you what to look for and how to interpret what you find. You are testing, extending, or applying a theory that is not yours.

A conceptual framework is one you build. It draws several theories, concepts and prior findings together into a structure that fits your particular study, usually because no single existing theory covers it. You are constructing an argument about how the pieces relate, and you must justify every connection. This is hard to manage in practice, and some universities discourage it without prohibiting it outright. Why? Getting pieces from different theories to fit properly is genuinely difficult.

A quick test: are you testing a theory, or building an explanation? Testing points to a theoretical framework. Building points to a conceptual one.

The burden sits in different places. With a theoretical framework, it is in choosing one that will actually carry your research questions — demanding, but comparatively straightforward. With a conceptual framework, you must justify the structure itself, because you built it, and a committee will ask why each piece is in there. It is far more intensive and carries far more risk.

Which you need is not a matter of preference. It is set by your field, your program, and sometimes by one member of your committee. Ask directly and get it in writing, because the two demand different approaches in the relevant chapters, and discovering the mismatch late is expensive.

If you find this confusing, so does the literature. The terms are widely used interchangeably and incorrectly, and most doctoral programs never teach framework selection at all — which is why candidates usually meet the requirement for the first time when a committee asks for something nobody has shown them.

A worked example, at the institution many of our candidates attend. Capella's chapter guides do not use conceptual framework at all. They call it theoretical orientation — the scientific perspective from which the research is conducted — and they state that a theoretical framework may comprise a number of different theories, provided the constructs borrowed from them are compatible and deal with the same material.

So at Capella the distinction above is not the operative one. Multi-theory work sits under the theoretical heading, and what the wider literature would call a conceptual framework is handled there. Looking for a section called "conceptual framework" in their guidance will find you nothing.

Their warning is the useful part, and it applies anywhere: do not blur or blend frameworks unless they can be authentically integrated and are needed for the study. If you use more than one, you must show how the concepts work together and can validly be considered together. That is the real burden, and it is what a committee will test.

One more figure from the same guidance: the theoretical orientation section is typically two to four pages. Short. It is a statement of the perspective you are working from, not a literature review of every theory you considered.

One warning. If you are being told "theoretical framework" by one person and "conceptual framework" by another, that is not a vocabulary problem — it is a disagreement about your study, and it will not resolve itself. Take it to your Chair to arbitrate. The same check from the comment "Your framework doesn't fit" is on What Your Chair's Comments Actually Mean.

The framework trap nobody warns you about

You choose a framework at proposal stage, the proposal is approved, and the framework is approved with it.

Then your thinking moves. That is not a failure — it is what doing the research is for. But few candidates have everything settled at proposal stage, and later thinking routinely shifts the thesis or the research questions. When it does, the approved framework may no longer fit.

And it is now locked in. Changing it means going back to a committee that has already signed it off, on work that has already been built on top of it. This is one of the most reliable sources of serious downstream trouble in doctoral work, and it is entirely invisible at the point where it is created.

What to do about it. Before the proposal is signed off, ask yourself honestly whether the framework can survive your questions changing. If your research questions are still moving, the framework is not ready to be approved — and saying so at proposal stage costs a conversation. Saying it six months, a year, or two years later costs considerably more.

The three things that have to fit each other — and you

This is the part that causes the most damage, and it is worth being slow about.

This is the part where real damage happens. Read the following with all due care.

Your research questions, your theoretical framework, and your survey instrument have to align with one another. Not loosely. Precisely.

• The framework determines what counts as an answer. It defines your constructs and tells you what relationships are worth looking for.
• The instrument has to measure those constructs — the ones your framework names, not adjacent ones that sound similar. And you have to be able to get it. Validated instruments belong to whoever developed them, and using one means asking their permission. That is usually granted, but it is not automatic and getting that permission takes time.
• And the research questions have to be answerable by the analysis that the instrument's data supports.

Break any one of the three and the other two stop working. A validated instrument measuring the wrong constructs produces clean statistics that answer a question you did not ask. A framework with no available instrument leaves you unable to collect anything. And research questions that outrun your instrument leave you with nothing to say at the end. If your questions ask about something your instrument does not measure, your Chapter 4 results cannot answer them — so your concluding chapter has to either claim more than your data supports, which a committee will catch, or admit the study did not answer its own questions, which is worse.

And there is a fourth element nobody warns you about: you.

All three have to be doable by you, in your circumstances.

• Cognitively. Can you actually perform the analysis this framework and instrument require? Structural equation modeling is a superb choice for somebody trained in it and a year-long detour for somebody who is not. The same is true of phenomenological analysis — as this page explains, it is a trap for the unprepared.
• Financially. Survey platforms cost money. At the volumes doctoral research needs, Survey Monkey and its equivalents are a real expense — and many candidates have not budgeted for one. Statistical help costs money. Coaching help costs money. And extra terms cost the most of all. You bear every one of these. Beware a research design that appears highly feasible — it may well be, but only at an expense you cannot bear.
• In time. Recruitment takes longer than anyone plans. Interviews take longer to write up than to conduct. Coding takes far longer than candidates expect. A design that works in principle and takes eighteen months you do not have is not a workable design — and the overrun is rarely just time. It is additional terms, at full tuition, with your funding window closing behind you.

Which brings you back to the research questions.

You cannot properly appraise a framework or an instrument without knowing precisely what you are asking. And you cannot finalize your questions without knowing what a framework and an instrument would let you ask. The three are settled together, iteratively, or they are not settled at all — and the fourth element, your own capacity, constrains every round.

This is slow, unfamiliar, painstaking work, and it is uncomfortable precisely because the penalties for getting it wrong are so high. It is also one of the places where coaching help makes the largest difference we ever see.

It is the highest-value work in the entire dissertation process. Be warned.

What to do instead

• Use it to find, not to decide. Generate search terms, then go to the databases.
• Read the research that used the framework. Not summaries of the framework. The studies.
• Ask your Coach, your Chair, or your statistician, in writing, whether the framework and the analytic approach fit the questions you have ended up with.
• Ask them again if your questions sharpen as you read — and they should. Indeed, if they do not, that is an early warning sign of trouble ahead. Questions that never change usually means the reading is not going deep enough.
• And keep asking the AI to argue against you. Not once — as a habit. Every time you settle on something, ask it for the strongest case a hostile reviewer would make. Inverting the question is the only reliable way to get useful work out of a tool built to agree with you.

Full treatment of the agreement problem, with a test you can run on any answer you are given, on our Human-in-the-Loop Structural Refinement page and AI: Use & Abuse.

How many participants do you actually need?

More than you think, and the number is not a matter of opinion.

Quantitative and qualitative work answer this question completely differently. Both are below — quantitative first, qualitative immediately after.

Quantitative: it is a calculation

Run it at proposal stage. Software does it in minutes — G*Power is free and standard.

Four things go into it:

• The statistical test you intend to use
• The effect size you expect — from the literature, not from hope
• Your significance level — usually 0.05
• The power you want — convention is 0.80, meaning an 80% chance of detecting the effect if it is really there

All four come from your design and your literature. None of them is a preference.

And here is the part candidates discover too late.

An underpowered study cannot answer its own research questions. Not might not — cannot. You will collect data, run the analysis, find nothing, and be unable to say whether nothing is there or whether you simply could not see it.

That is not a null result. That is an uninterpretable one, and it is considerably harder to write a discussion chapter around.

Qualitative: saturation, and it is still not arbitrary

Saturation is the standard — you keep sampling until new participants stop producing new themes.

But your proposal has to state a target before you begin, and your committee will expect it justified. The justification comes from comparable published studies in your field, not from a number you liked.

And "I reached saturation at twelve" is a claim you must be able to evidence. You will need to show what stopped changing, and when.

What to do when you cannot recruit enough participants

This is one of the commonest crises in doctoral work and almost nothing is written about it.

It usually arrives months after approval. The design was approved, the ethics cleared, the instrument ready — and the people will not come. Response rates in the single digits. Organizations that agreed and then went quiet. A population smaller in practice than it looked on paper.

Deal with the honest question first

Was the population ever large enough and reachable?

A design that requires participants you cannot get is not a workable design, and it does not become one by trying harder. That is the same alignment problem this page describes — arriving late, and expensively.

What actually works

• Extend the timeframe before you extend anything else. Recruitment is nearly always slower than planned, and more time is the cheapest fix available
• Widen who counts as eligible — but only if the people you let in can still answer your research questions.
  An example. Your study is about nurses with more than ten years of experience. Recruitment stalls, so you drop it to five years. That may be fine. But if your research question is about how long experience changes practice, you have just admitted people whose answers cannot address it — and you will not find that out until you analyze the data.
  The test: before you widen, ask what the new people would say, and whether it answers your question. If it does not, widening buys you numbers and costs you the study.
• Change the recruitment route. Gatekeepers, professional associations, snowball sampling through participants you already have. Direct approach is frequently the weakest channel and the first one candidates try
• Add sites. Two organizations rather than one, if your design permits it
• Reconsider the incentive, where your ethics approval allows

And the option candidates resist

Change the design.

A qualitative study with fifteen participants is a real study. A quantitative study with fifteen is not — it is an underpowered study that will not answer its own questions.

If the numbers are not achievable, a methodological change is not a defeat. It is the correct response to what you have learned about your population, and it is far better made now than defended at a defense.

Take it to your Chair early. An amended methodology is a process. A study that quietly proceeds underpowered is a Chapter 4 that cannot be written.

How do I choose a research instrument?

Three separate questions, and candidates routinely check only the first. Take them in order — each one can stop your study, and the later ones stop it later.

1. Is it validated?

Candidates check the first and discover the other three much later, and often too late.

Do all four at proposal stage. Permission alone can take months, and it can be refused — which is survivable in month two and not in month eight.

Validated means it has been demonstrated to measure what it claims to measure, in a peer-reviewed process, with published psychometric properties.

Not: it looks sensible. Not: somebody used it once.

And validation is population-specific. An instrument validated on undergraduates in one country may not be validated for mid-career professionals in another. Using it outside its validated population is a limitation you must state — and a committee member will ask about it.

2. Are you permitted to use it?

Instruments belong to whoever developed them. They are real intellectual property. Using one nearly always means asking permission.

That is normally granted, but it is not automatic, and it takes time — depending on how overloaded the originator is, sometimes a lot of time. Some require a fee. Some require you to share your data. Some are simply unavailable.

One thing worth knowing when you ask. The originator benefits if their instrument becomes widely used — citations, validation in new populations, and standing in the field. You are not only asking a favor. Say what you intend to study and where you intend to publish, and the request becomes a proposition rather than a petition.

Ask early. A candidate who discovers at month eight that their instrument cannot be obtained has lost the months and frequently the design with them.

Permission to use is not permission to publish

Two separate permissions, and candidates routinely secure the first without knowing the second exists.

Educational fair use does not cover a copyrighted instrument, because an instrument is treated as an entire work rather than an extract from a larger one. So you need permission to use it in your study at all.

Then, separately, you need permission to reproduce it in the published dissertation. Capella's Doctoral Publications Guidebook (version 7.2, April 2021) states it directly: having permission to use an instrument in your research does not necessarily mean you have permission to publish it in the dissertation.

Start the permissions process the moment you know you will need it. It can take weeks, and publication can be held up while it is sorted out.

3. Can you build your own instead?

Usually the answer is no, and you should be glad of it.

Many programs — most online programs — prohibit it outright for doctoral work. And where it is permitted, developing and validating an instrument is a dissertation in itself. It is not a step within your study; it is a different study.

If your research questions can only be answered by an instrument that does not exist, that is important information about your questions. Take it seriously at proposal stage rather than discovering it later.