PDA

View Full Version : Benchmarking or social engineering


WingNut60
27th Jan 2018, 15:08
Aussie teens set to be tested on cultural tolerance

Australian teenagers would be tested on their tolerance of other cultures and ability to identify fake news under a new international assessment this year.
The Organisation for Economic Co-operation and Development will measure 15-year-old students’ “global competence” at the same time they are assessed on their reading, maths and science skills as part of the Program for International Student Assessment.
Results of the PISA tests, which students sit every three years, are used to compare education standards across the world.
The new test, which covers issues such as racism, prejudice and global warming, has been designed to find out how well schools prepare students to work with others from different cultures and beliefs.
It will measure students’ ability to analyse news articles critically and how well they understand different cultural perspectives.
A separate questionnaire will collect data on students’ attitudes towards people from different backgrounds.

Political questions with subjective assessment.

.... but


So far, 44 countries, including the US, Britain and several European nations, have withdrawn their schools from the test.
Australia, Canada, Scotland and Singapore are among 28 countries listed as taking part.

Gertrude the Wombat
27th Jan 2018, 16:01
Well, a prerequisite for evidence-based policy making is actually gathering some evidence. Which sometimes shows that there isn't a problem so no policy change are needed.

But in our part of the world basing policy on evidence (y'know, "facts") is deeply unfashionable these days so it's not surprising we're not joining in.

jack11111
28th Jan 2018, 05:01
As long as the results are in NO way connected with the test-taker, I don't see a problem. Government needs the views of the citizens to help make policy.

ChrisVJ
28th Jan 2018, 05:16
A test has to test an ability. This part of "the test" is really a survey. The problem I have with this is that a "test" or a "survey" of this type with such an obvious hoped for result is in itself 'mind forming.' It is not teaching kids how to think but what to think.

The results of the PISA tests are not used to make government policy except in forming education. It is, generally, a well accepted test but like all tests it is influenced by who the testers are and what they want to learn.

WingNut60
28th Jan 2018, 06:02
A test has to test an ability. This part of "the test" is really a survey. The problem I have with this is that a "test" or a "survey" of this type with such an obvious hoped for result is in itself 'mind forming.' It is not teaching kids how to think but what to think.

The results of the PISA tests are not used to make government policy except in forming education. It is, generally, a well accepted test but like all tests it is influenced by who the testers are and what they want to learn.

Encapsulates my reservations exactly.
My understanding is that the PISA tests are supposed to be about benchmarking delivered education.
That can only be achieved through objective assessment.

How querying "attitudes" towards politically loaded topics can be executed objectively is quite a puzzle to me.
And how that same assessment method can be applied to some of the third -world countries which already struggle with abysmal PISA results is similarly puzzling.

It makes me inquisitive to know the EXACT reason(s) that the UK, US, et al have opted out.

Mostly Harmless
28th Jan 2018, 21:34
I do agree with half of this, the part wherein they teach kids how to tell the difference between a fake news and a real news story. This might sound silly but I have seen studies where they were checking kids on this and the teenagers couldn't tell the difference between a news story and advertising that was disguised as a news story. It's not always politics and I think having a somewhat more savvy critical thinker is a good thing.

ChrisVJ
29th Jan 2018, 07:08
MH
Our education system is trying to manage a transition from teaching "Knowledge" to "skills" or "competencies," Critical thinking, innovation, collaboration, learning (as a skill). One of the major problems with this is working out how you assess these skills. Our district is probably one of the leaders (among school districts) in the state system but we are spending an inordinate amount of time and effort on 'assessment.'

For instance how can a test such as PISA assess innovation or creativiyty? The answer from a teen might well appear to have little to do with the question yet, like many innovations, be the result of "left field" thinking.

Gertrude the Wombat
29th Jan 2018, 18:11
For instance how can a test such as PISA assess innovation or creativiyty?
From Richard Feynman's description of trying to teach physics to kids in Brazil (wasn't it?) one might form the opinion that at least trying to measure such things is of value.

Mr Optimistic
29th Jan 2018, 21:32
And how should we test the objectivity of the marking? If they agree with me, pass. If they don't, fail. Just like promotion in the BBC.

WingNut60
29th Jan 2018, 23:18
...... If they don't, fail. Just like promotion in the BBC.

A good analogy.

A previous employer in Indonesia had an employee assessment program of about 10 assessment categories of which the majority were entirely subjective: honesty, creativity, trustworthiness, etc.

In the wash-out it was all meaningless and advancement was based on ethnic origin, religion and connections.

Around Y2002 the system was revised to introduce objective assessment of Key Performance Indicators within Key Responsibility Areas.
Following that, it came to be that advancement was now dependent on ethnic origin, religion and connections.

India Four Two
30th Jan 2018, 00:04
WingNut60,

I've had a similar experience. About 10 years ago I was a middle-level manager in a western company, based in an asian country.

In a fit of ludicrous enthusiasm, the General Manager decided to implement western-style performance appraisals, including self-assessment prior to a sit down with a manager.

My ex-pat staff all assessed themselves quite fairly and in line with my assessments of them. My local professional staff assessed themselves, without exception, as Exceptional.

WingNut60
30th Jan 2018, 00:50
....... ex-pat staff all assessed themselves quite fairly and in line with my assessments of them. My local professional staff assessed themselves, without exception, as Exceptional.

Oops, sorry. I had forgotten that bit. My experience precisely.
And then sitting down with them and trying to explain, politely of course, that their performance was not quite as exceptional as they seemed to think, and why.

It's at these times that Asian cultures show their lingering links to strong class / caste systems.
Western cultures less so.

meadowrun
30th Jan 2018, 01:24
It's at these times that Asian cultures show their lingering links to strong class / caste systems.
Would being assessed/evaluated by a white man (heaven forbid a women), in life beyond university, have anything to with it?

ChrisVJ
30th Jan 2018, 01:33
Gertrude,

As mentioned we, and the rest of the province, are working on ways to assess the new skills.

For instance we might use a series of criteria, a) Includes new ideas not discussed in class. b) Includes a development of an idea discussed in class, c) Discusses an idea discussed in class d) does not include anything new.

This way we assess the answer without requiring a particular direction

Each part of innovation might be parsed and assessed, ie. Idea, discussion, extrapolation, relevance , feasability, description,.

A few years ago we got a grant for some teachers to conduct a program in which they tried to create consistency in marking English essays. Three teachers across three grades in one school. Over several months they achieved some consistency but not enough to say we had it licked. . . . . . . . . and the province has abandoned final grade 12 exams for university entrance except for English. Education is indeed an interesting world.

WingNut60
30th Jan 2018, 01:54
Would being assessed/evaluated by a white man (heaven forbid a women), in life beyond university, have anything to with it?

Possibly, but not to any great extent, no.
Most of the assessment were carried out by Indonesian managers. My small crew formed only a tiny part of the overall system which was mainly indigenous.

Local employees were quite forthcoming in letting the expatriate managers know that they preferred the assessments carried out by us.

Expatriates were not, at that time, subject to that assessment process.
Not performing? Your gone. Simples.

parabellum
30th Jan 2018, 02:05
If it is the Ballard style question and answer format, (question then up to four possible answers), the outcome of a survey can be completely skewed by the questions asked and the answer options.