I once had an online 'discussion' with a chap claiming that
the twin towers were, in fact, brought down by the US government and that 9.11
was all an inside job. He sent me a link to, in his words, a ‘peer
reviewed academic journal article’ to back this theory
up.
The link led to the ‘Journal of 9.11 studies’. One of the editors was
Kevin Ryan who coincidentally was also the article’s author. Editors do
occasionally write articles for their own journals but still...I was dubious. So
I googled Kevin Ryan and found that he’d written
a book on why 9.11 was actually 'an inside job' and
was fired from his previous university job his views. The article was about
Nano-thermites (which he said the government used to blow up the twin towers), a type of explosive, and yet Ryan was working on water testing.
Things didn’t add up.
O Red Flags
Imagine going to a restaurant and seeing no customers, the
paint peeling and a smoke coming from the kitchen. None of this necessarily
means the food there is bad, -hey it might be great, but these are all the kind
of things which Dorothy
Bishop refers to as ‘red flags’. These red flags
can acts as a kind of early warning system.
This brings me a conversation about the relative pros and
cons of learning styles/MI and the Montessori Method I had on twitter earlier
in the year. I was sent a link to a series of educational articles by a teacher
who thought they supported her case. When I read them i noticed a number of red flags. I
thought it would be useful to blog about these and hopefully this
will be useful for other teachers assessing the quality of sources. So here are
a few things to look out for.
O Mode of publication
Is the article in a book or on a website or in a
journal?
This matters because anyone can say anything in a book or on a
website. Books often seem impressive as if somehow putting something in a book makes
it more weighty and serious. Usually, academic journals, which have been
through peer-review, are more likely to have credible information than books. Websites
are almost always a no-no as there is usually zero quality control. Disclaimer
(of course it always depends on exactly what you are looking for and what kind
of website/book it is).
What about the papers that I was directed to look
at? They seem to be on a website, which is an initial red flag.
However, when we get to the articles in the download section they actually seem
to be from real journals, namely, the MASAUM
Journal of Reviews and Surveys (volume 1), MASAUM Journal of Open Problems in Science and Engineering (volume 1) and the International Journal of
Engineering Research and Applications (volume 2); so far so good.
But O (first red flag) why publish education
articles in journals of engineering?
O Dubious journals
So these
articles are in journals but what kind of journals?
Ideally the journals would
be Peer-reviewed, and well established. The first two of these journals fail the
‘well-established’ test as they are all only seemingly on their second or first
volume and are not available anywhere on the web. The third one actually does exist and looks (roughly) like a journal should. The site looks a bit cheap
and unprofessional (Gmail address for submissions etc) but it exists, which is
a start.
O Pay for play
A little bit of digging around
the FAQ section of the third journal and we find this:
Q: How much do I have to pay for publication fee
A: $150
Paying for publication isn’t a great
sign. As it’s an open-access journal (anyone can view the articles for free) this
doesn’t necessarily mean it’s a bad thing, as long as the quality control isn’t
affected. If we go to the ‘stats’ page though, we can
see that they accept roughly 40% of all submissions. This is quite high considering
journals like ‘Applied Linguistics’ accept only
about 10% of submissions. It’s also notable that they record stats monthly and from
those we can see they accept about 250 papers each month which means each volume has well over 300 articles. For context that’s
about 7 years’ worth of ELTJ articles in one volume.
O Peer-reviewed
Peer-review is sometimes mistakenly thought of as
the ‘end’ of the process. But actually all peer review means is that a couple
of other people in the profession have read it and think it’s good enough to be
published. This doesn’t mean it’s perfect, or that no one can question it, just that it has reached a certain level of acceptability. After peer-review,
the academic community at large get their teeth into and then we often see
criticisms, repeat studies and sometimes retractions. The International
Journal of Engineering Research and Applications is actually peer-reviewed so that
seems pretty reassuring. Except, considering they boast that peer-review only takes 4-6 days (not the couple of months
journals usually take) and because they read about 800 papers a month and only have about 27 reviewers,
they are getting through those papers at an astonishing rate! You have to worry
about quality control.
O The author
If you’re at all in doubt you can always check out
the author’s credentials. According to his website Dr. Qais Faryadi (or ‘Dr. Prince’ as his website calls him) has a PhD in computer science
and a master’s in Sharia Law as well as being an expert in curriculum design,
criminal law, software engineering and Islam. He lectures on all
these subjects and has even published a number of books (or rather ebooks) on
such diverse topics as teaching, Islam and “Magnesium The Health Restorer: The Missing
Link To Recovery”.
O The nose-test
When you actually start reading
the paper, does it sound plausible or like an academic paper should?
Dr.
Prince’s starts his learning styles paper with the statement that:
This evaluation examines teaching and
learning from the lenses of mind blowing scholars such as David Kolb,
(1984), Honey, (1982), Dick and Carey model (1990), Anthony, Sudbury Model, VAK
Model and Madeline. (Faryadi, 2012:222)
Language like this should set
alarm bells off instantly. This paper isn’t going to be an unbiased review of
learning styles, not when the writer believes their creators to be ‘mind blowing’.
Not only this, the article is littered with errors. In this small section alone
we have such oddities as:
- ‘Honey’ should be ‘Honey and Mumford’ (as in the reference section),
- He starts by talking about ‘scholars’ but then switches to ‘models’
- Anthony? Who he?
- Madeline? Who she?
Wouldn’t peer-review usually sort
errors like these out? Then again, with reviewers getting through 50 or so
papers a month in 4-6 days each, it’s perhaps not surprising that the quality
suffers.
The article itself is a relatively
unremarkable laundry list of things the author believes ‘good teachers’ should
do. There is no attempt to critically engage with the various learning styles
models presented or to talk about why and how they differ from each other.
So in conclusion we have sloppy education
articles written by an expert in computer science and law who claims expertise
in a number of other fields. They were published in science and engineering journals
that either no longer exist or have very little reputation. The one journal
that does exist, charges for publication and has numerous credibility issues.
The food may be great here, it really
might…but I’m going to look elsewhere.