Research Evaluation Tools for Articles in the Social Sciences (DRAFT)

UPDATED on September 9, 2013:

These tools were again revised, with the current version on my Research Tools page.
Feedback is always appreciated.

UPDATED on August 21, 2013:

Based on some very helpful feedback, I revised these two tools:

As before, feedback will be most appreciated.

ORIGINAL POST:

I am in the process of creating 2 tools to help my graduate research students assess and evaluate research studies, and am interested in getting some feedback on them. They are:

While I have seen various tools for specific purposes, I have not seen many that were intended for general use in the social sciences. Furthermore, while these cannot be applied to every qualitative or quantitative study in the social sciences, they are intended to be applicable to most of them.

Do these work? Are they helpful? Is there anything major missing or that should be combined, edited, or refined? Any feedback at all will be most appreciated.

Once I finalize these, I will make them freely available under a Creative Commons license.

 

I Completed My Analysis

I am happy to say I have just completed the analysis of my data for my doctoral thesis!

Let me clarify what I mean. By analysis, I mean making sense of the 23 interviews I completed by coding them, grouping similar concepts together, and then putting these concepts in a coherent order to present for my readers. That may not sound like a lot, but with hundreds of pages of interview transcripts and over 1000 codes to navigate and organize, it is a significant  accomplishment.

While I have written up my analysis along the way (cf. Richardson’s work on writing as a method of inquiry), I hope to have my full draft analysis completed in another week or so. As I am engaging in narrative inquiry, this will be, in all likelihood, my longest thesis chapter.

I just sketched a tentative timeline of thesis work for the next week, so will keep my fingers crossed to maintain its trajectory (which I will do via Twitter).

 

Here comes the Analysis!

I have neglected my blog for a bit recently, though the (good?) reason is because I have worked so intensely on my thesis! While I continue to tweak and clarify and refine my first chapters, something that I fully expected to still be doing, I am nearing ready to send the first round of my analysis for review (with the target of Wednesday, 12 September). While I do not envision being finished with it by then, I expect to have enough of substance to get some real feedback on it. I find that having a concrete goal and date is helpful with this experience in endurance!!

Between now and then will find me finish making all the corrections and edits that have previously been identified, continue to refine my literature, and adjust internal issues of consistency and repetition. Phew!

 

Research Seminar this Wednesday!

I am presenting my doctoral thesis research in progress at my department’s Seminar Series. I often attend the Department of Educational Research seminars from a distance, though this will be the first time I have presented at one.

While this has been a great opportunity to help me focus my writing and make some solid progress, the most interesting thing for me will be that this is the first time I will discuss my research holistically. Thus far, I have only spoken with other people about various elements of it; this will be a glimpse at the entire study.

I wonder what this experience is like for others? If you engaged in doctoral research, what was your experience like the first time you presented your study for feedback and comment?

 

Why is there not a single Doctoral Thesis outline or structure?

After working with the transcription for my doctoral thesis, I have found creating the structural outline for the entire write-up to be the greatest challenge.

Now, don’t shake your head wondering why I am considering this. Indeed, we always need an intro, and some literature, methods, data analysis, and the like. The trick I have found, least for those of us working on a doctoral thesis or doctoral dissertation, is to determine the structure of this given that we are conducting original research.

There is the main issue–this is ORIGINAL research. Why (or how?!) can my work follow the same structure as somebody else’s? Yes, we somehow need to address some of the same things, but our research designs and personal interests and writing skills all come into play. Nobody (I hope!) would confuse a randomized control trial with an autoethnographic inquiry, so why (or how?) should they look or be structured the same? My research engages in narrative inquiry (with a healthy dose of actor-network theory), and as my interpretation will be presented alongside the analysis itself (at least in some places), the thinking about using a cookie-cutter approach to chapters and sections vanishes. For those of us who relish in the originality while still wanting to follow a map, this part of the process can certainly cause some concern.

While an Introduction usually precedes Data Collection, consider that there is not only a single way to organize all of this. Some of these elements, such as the researcher’s ontological stance, disciplinary frame where the research is situated, and significance–to name only a few–can be placed in different locations, as needed. At times they may not even need to be isolated (or even mentioned); again, it depends on your work . . . and original work means there is no standard outline that works best for everybody; you have to make it up yourself.

BTW, don’t forget to be prepared to explain why you did so!