Chapter 3: Methodology: 'Ringing the bells'

What does Chapter 3 have to prove?

The methodology chapter is mostly quite formulaic. Your school will likely have quite specific requirements, whether formal or informal. Most chapters need these.

• A synopsis of the study — its purpose and the background that matters, expanding on what you wrote in Chapter 1
• Your research questions and hypotheses
• Your theories and theoretical framework, set out properly
• Your research design, including your population and sample parameters
• An ethics section — how you treated your subjects
• A complete outline of your methodology, naming the instruments you used
• Some discussion of validity and reliability

What is Methodology?

Vague research ideas

The problem is that many doctoral candidates discover (often to their horror!) that the methodologies they actually need to use to perform the research they have committed to are not the ones that they got approved at the proposal stage and echoed in their introductory chapter. Universities do not like candidates going back and changing their proposals and even if your university will allow it, it will likely take a great deal of time and set you back badly. The alternatives? Twist your statistical approach into a pretzel to try to make it fit (…a really good and devious statistician can help with that…) or abandon the thesis and your doctoral dream. The time to think and think hard about your theoretical framework and its accompanying analytical approaches whether qualitative or quantitative should be while you are writing your proposal. If you are doing survey-based research, think about the questions you may want to ask and what answers you might expect? How will those be analyzed?

Surveys

General questions, responses that have been allowed to go off-topic, and softball responses, let alone missed or incomplete responses can make qualitative analysis a painful nightmare. It is a good time to get help. At minimum try out some questions and ask yourself what the answers might be and then how they would be analyzed.

Quantitative & Qualitative

For both quant and qual theses look carefully at multiple examples of published scholarship that use your frameworks and approaches and make sure you really understand why they asked the questions they did, how they framed those questions, and how they analyzed them. Qualitative methodologies lend themselves to first-time users getting lost in the methodology. Be very methodical, stick to one approach and get help early if it suddenly looks like you have gotten yourself into a nasty analytic tangle from which you will not emerge.

What your study can actually establish

Three words get used interchangeably and mean three different things.

Correlation. Two things move together. When one is high the other tends to be high, or tends to be low. That is the whole claim. It says nothing about which came first, or whether one produced the other.

Covariance. The raw measure of how much they move together and in which direction — positive when both rise, negative when one rises as the other falls. You will report correlation instead, because correlation is standardized and comparable across studies while covariance is in whatever units you measured.

Causation. One thing produces the other. This needs three conditions: the cause comes before the effect, the two are related, and every other plausible explanation is ruled out. The third is what stops nearly everybody.

If you are running factor analysis, structural equation modeling, multilevel models or anything multivariate, you will meet the word covariance in a second sense — a covariance matrix, the table of how every variable in your model moves with every other one.

It is the same quantity, arranged as a grid. Every variable appears down the side and across the top; each cell holds the covariance between that pair, and the diagonal holds each variable's variance with itself.

Why it matters to you: it is what your model is actually fitted to. Structural equation modeling in particular does not analyze your raw data — it compares the covariance matrix your data produced against the one your model predicts. A good fit means those two matrices are close.

Which is why your Chair may ask about it directly, and why "did you check the covariance matrix" is a question about whether you looked at your data before modeling it. Wildly different variances across variables, or two variables almost perfectly related, will distort a model without producing any obvious error.

None of this changes what you may claim. A covariance matrix is a description of relationships, however elaborate the model built on it. It is not evidence of cause.

Now the part that decides your conclusions.

A cross-sectional study collects all its data at one point in time. A survey. A set of interviews. A dataset at one point in time. Records that already exist. You measure everything once.

Nearly every doctoral study is cross-sectional, and it can show that two things go together. It cannot show that one changed the other, because you never watched anything change.

A longitudinal study measures the same people, organisms or phenomena again and again, over months or years. That is the only kind of study that watches something develop.

Almost no doctoral candidate runs one. They take years you do not have, usually massive funding, and — assuming humans as subjects — participants who stay reachable throughout. A time-constrained candidate with no research budget is not going to follow two hundred nurses for a decade.

An experimental study assigns people at random to different conditions and controls what else happens to them. This is the only common study that can support a causal claim — and outside psychology, education and medicine, few doctoral candidates are in a position to run one either.

So state the limitation and state it early.

A cross-sectional study is a standard and expected shape for doctoral research. Your committee knows it. What they will not accept is a cross-sectional study written up as though it showed change.

Put it in this chapter, not just in chapter five. This study is cross-sectional: all variables were measured at a single point in time, and it therefore cannot establish causality.

Writing that sentence now saves the argument later. A methodology chapter that names its own limitation is a chapter a reviewer trusts.

And a trap worth knowing before you collect anything.

Nurses with twenty years of experience report higher satisfaction than nurses with two.

That is not satisfaction rising with experience. The twenty-year nurses are the ones who stayed. The dissatisfied ones left years ago and were never in your sample.

A cross-sectional study cannot see who is missing. If the thing you are studying affects who remains available to study, your sample is telling you about survivors rather than about the population.

Write the main claim you hope to make, and look at the verb.

Increases, improves, leads to, results in, produces, drives, affects, develops over — these are causal claims, even though none of them contains the word cause.

Is associated with, is related to, differs between, is higher among — these are what a cross-sectional study supports.

If the verb you want is in the first list and your study is in the second, you have found the problem now rather than in your defense.

Don't get stuck marking time!

Chapter 3 – Dull? Yes. Formulaic? Yes. A good place to deploy AI? Mostly yes.

Where AI actually earns its keep — and where it does not

Methodology chapters have lengthy lists of required elements. Design, population, sample and sampling method, instrumentation, data collection procedure, analysis approach, ethical considerations, validity, reliability, limitations. Different programs may order them differently and use different names, but the list is finite and very concrete. It is in your handbook or your template if your institution uses either or both. If your institution uses neither? Ask your Chair what is acceptable.

Checking your draft chapter against a finite list of the elements your program requires is mechanical work — precisely what AI tools excel at.

What it does well here:

• Completeness checks. Paste in your program's required element list, then your draft. Ask what is missing or thin. It will find gaps you have read straight past twenty times — and you can verify every one against the list in seconds.
• Terminology drift between your introduction and your methodology chapter. Same variables, same framework, same population description — or they should be. Ask for a list of terms that differ between the two. Then go and look at each one. Chairs and committees notice this. You will not, because you know what you meant.
• Instrument-to-question mapping. Upload your instrument and your research questions and ask it to map each item to the question it serves. Then check the map yourself. What you are looking for is research questions with no items behind them, and items measuring something you never asked about. Both are common and both can be costly to fix later.
• The sample size arithmetic. For the designs most candidates use, a power calculation is standard arithmetic — effect size, alpha, power, in; required n, out. AI can do it and you can check it against any power calculator in two minutes. Doing the arithmetic is not the same as judging whether the result is adequate — see below.
• Tense. AI can change future tense to past tense in a flash, in any text. However, note that copying and pasting text directly from your proposal is contra-indicated. Paraphrase at minimum — and see Chapter 1 on why a paraphrase can quietly change what you committed to.
• Reverse-outlining. One line per paragraph saying what it does. Then check that against the structure your program requires.

Under no circumstances use AI for:

• Analyzing whether your method is appropriate. Give it your instrument, your sample, your entire methodology chapter, and ask. It will say yes. It says yes to nearly everyone — see Chapter 1 for the research on why. The problem is not how much it knows about your study. It is which direction the answer points.
• Judging your validity and reliability claims. It can tell you what construct validity means. It cannot tell you whether your instrument has it.
• Judging whether your sample size is adequate. It will happily calculate n from whatever effect size you hand it. Whether that effect size is realistic in your field, whether you can actually recruit that many people, and whether your recruitment plan survives contact with reality are questions for your Chair, your coach, or your statistician.

And a specialist tool does not change this. There are now AI tools built specifically for academic research — Elicit, Consensus, SciSpace, NotebookLM, Paperguide and others. The ones that retrieve from real paper databases are genuinely better at one thing: they invent far fewer citations, because they are pulling from actual indexed papers rather than generating plausible text. That is a real advantage and worth having.

It is also a solution to a different problem. Grounding a tool in a paper database stops it fabricating references. It does not stop it agreeing with you about your own methodology, because that is not a fact to be looked up. It is a judgment about your work.

More on that distinction on our Human-in-the-Loop Structural Refinement page.

One warning about AI and this chapter specifically.

Your methods section is dense with sentences that look redundant and are not. Two specifications of the same object, phrased similarly, doing different work.

We have documented a case where a current model read two such sentences as one duplicated sentence. Cutting either would have made the study unreproducible, and the remaining prose would have read perfectly well.

Whatever else you use AI for, do not let it tighten your methods.

Just realized your methodology does not fit?

It is not a disaster. Yet. What you do about it is critical, however.

Why it happens, and how to avoid it next time, is discussed on our Methodology at Proposal Stage page. You are likely well past that point if you are reading this page. What matters now is what your approved proposal actually is — and what it is not.

It is a binding agreement, and it does protect you. Up to a point.

Institutions describe it in almost those words. Utah State's School of Graduate Studies calls the approved proposal "a binding agreement between the student and the committee regarding expectations for the final thesis, project, or dissertation." One University of Tennessee at Chattanooga program handbook puts the protection directly. Assuming you carry out the project as described in your proposal, the committee cannot find your defense unacceptable based solely on the research plan.

How far that protection actually goes varies. Utah State states it as university policy; the Chattanooga wording comes from one program's handbook. Find out what your own program says, and get it in writing if it is not already there.

So follow your approved plan and your research design is largely out of play at defense. Deviate silently and you hand back exactly the ground you were standing on.

Here is why that protection is not enough on its own.

Your dissertation gets published. ProQuest, your institution's repository, indexed and searchable and permanent, with your name on it. Every hiring committee, every future collaborator, every journal editor, and every curious colleague can read it.

A weak research design that your committee approved is still a weak research design — sitting in a public database, under your name, for the rest of your career. "They said it was fine" is not a defense you will ever get to make, because nobody reading it will ask you.

Fix it because it is going to be published. That is the better reason, and it is the one that outlasts your defense.

And it is not going to slip past anyway.

A mismatch between methodology and research questions is precisely the type of thing Chairs look for. In the event it survives your Chair, you will then be running the gauntlet of your committee. And in the unlikely event it survives there, there is always still School Review.

Ignoring the problem is like ignoring a lump. It does not go away. It only gets worse, and your options narrow as time passes.

Most programs have a process for this. Changes are discussed with your Chair. Major changes require your supervisory committee's approval. Several institutions require a revised proposal document, or a written summary detailing every change, circulated for approval. At institutions with a scientific merit or research review stage, a substantial change may go back through it.

Your options:

• Amend the approved methodology. Discuss it with your Chair, prepare the written summary of changes, get committee approval, and expect it to go back through any institutional review stage. Slow, painful, entirely survivable.
• Adapt within the approved approach, if the fit is imperfect rather than wrong. Then say plainly in the chapter what you did and why. Deviations disclosed and justified are a normal part of research. Deviations discovered are not.
• Reconsider your research questions — but only if they genuinely improved as you read. Not to make a mismatch disappear. Bending your questions to fit a framework produces a thesis that answers something nobody asked, and that thesis gets published too.

Four things that will complicate it:

• IRB. If your change touches human subjects — different participants, different instrument, different procedure, different consent — you need an approved amendment before you collect anything under the new design. Capella's manual is explicit: once IRB approval is granted, procedures and methods cannot be changed without IRB consultation and approval, and the change must also be approved by the Research Chair in your school.
• Data already collected. Anything gathered under the superseded design may not be usable. Find that out before you decide, not after.
• Time. Committee approval takes as long as your committee takes. Add any institutional review stage on top. Build it into your timeline honestly.
• Re-review or re-defense. Some programs require a proposal to go back through scientific merit review, or to be re-defended, if the change is substantial. Ask early whether yours is one of them.

Discuss any mismatch with your Chair — or, if you would rather think it through first, with us — and do it as soon as you have found the problem. Not next month, and certainly not after you have collected your data.

If you are at an online institution, there are more gates than you think

At a traditional university, your committee is largely the decision. At an online institution it is one stage among several, and most of the others are strangers.

Capella's PhD Dissertation Process Manual — the latest version the university publishes, 3.2, dated March 2019 — sets out sixteen milestones. We use it here because it is unusually explicit, not because Capella is unusual. A doctorate is a fairly standardized product, and the requirements that manual documents — an approved research plan, a merit review, ethics clearance, a format and publications review, a defense, a final institutional sign-off — are, in our experience, what most institutions require in some form. What differs between institutions is not the requirements. It is how many of them are named, numbered, and handed to somebody you will never meet.

Beyond your committee there is a Scientific Merit Review: a reviewer assigned by the school formally assesses whether the research plan is sound, and the manual instructs learners to expect "multiple iterations" before it is approved. Milestone sixteen is the Dean's approval of the final manuscript.

At Walden, approval requires the committee, a University Research Reviewer, the IRB, a Form and Style Review, the oral defense, and finally the Chief Academic Officer.

None of these people know you. They are reading a document against a standard. A methodology that does not fit its research questions is exactly what that kind of review is designed to catch — and it will be caught later, by someone with no reason to be generous, rather than earlier by a Chair who wants you to finish.

Two Walden researchers put a figure on it. Comparing learners who took the dissertation support workshops against those who did not, Baltes and Brown report proposal approval rates of 71.2% versus 27.12% — roughly three in four unsupported proposals not approved. It is Walden faculty studying a Walden intervention, which is worth knowing, but it is peer-reviewed work in the Journal of Online Graduate Education rather than a marketing claim. Take it as an indication of how hard these gates actually are — especially if you are a non-traditional student, returning to university at mid-career, on a path you never envisioned when you were younger.