We should LOL @ this "no government" and "living free" guy's

UFOs, lost socks, discuss whatever you like here.

Moderators: Master_Kale, TNM Team

User avatar
bjorn98009_91
Silhouette
Posts: 688
Joined: Thu May 08, 2008 8:17 am
Location: Hufvudstaden, Sweden
Contact:

Re: We should LOL @ this "no government" and "living free" g

Post by bjorn98009_91 »

Jaedar wrote:It's interesting about that danish problem, cause in sweden we had the absolute opposite problem. Our grades used to go from IG, G, VG and MVG (not pass, pass, pass with merit, pass with lots of merit). However, teachers were handing out things like G+ and VG- all the time, so then they swapped it to 0-5 numerical adding two steps.

And now people are mad its to hard to get the highest grade :P. Also we very very rarely do multiple choice things in sweden, except for the university exam thingie, which is only multiple choice....
Well that's not quite true, now it's F to A. Same shit, different name.

But yes, I got mad at that extra step of granularity, but I understand the need. The most outrageous thing is the way the grading criterias are constructed. It's fucking impossible to understand what they mean when they write "Pupils describe briefly the meaning of the concept..." for grade E, "Pupils describe in detail the meaning of the concept..." for grade C and "Pupils describe in detail and with nuances the meaning of the concept..." for grade A.
Producer and Quality Assurance Manager for Deus Ex: Revision.
AEmer
Illuminati
Posts: 1490
Joined: Fri Jan 26, 2007 12:04 am

Re: We should LOL @ this "no government" and "living free" g

Post by AEmer »

This discussion of the grading systems is exactly the problem. Like, it's a fun problem and you can be pretty confident in the right way to do it, but clearly, nobody actually has a clue. It's this way all the way to the upper echelons who decide on the grading system. And it _shouldn't_ be this way because we have science that can actually let us decide on the optimal solution!
User avatar
Jonas
Off Topic Productions
Off Topic Productions
Posts: 14224
Joined: Sat Apr 24, 2004 9:21 pm
Location: Hafnia

Re: We should LOL @ this "no government" and "living free" g

Post by Jonas »

I'm not sure whether to be relieved or terrified that the only grading system I'm subject to anymore is measured in thousands of dollars and units shipped.

I mean I guess there's review scores as well, but they don't seem as important, in comparison...
Jonas Wæver
Chief Poking Manager of TNM

I've made some videogames:
Expeditions: Rome
Expeditions: Viking
Expeditions: Conquistador
Clandestine
DDL
Traditional Evil Scientist
Traditional Evil Scientist
Posts: 3791
Joined: Mon Oct 17, 2005 10:03 am

Re: We should LOL @ this "no government" and "living free" g

Post by DDL »

AEmer wrote:because we have science that can actually let us decide on the optimal solution!
Links?

I'd argue that it's very, very difficult to come up with a generic approved way of testing "aptitude" when aptitude is itself so incredibly context-specific. "Knowledge" is perhaps easier to test, but simple ability to retain facts is not necessarily a key determinant in subsequent success or suitability for a given job.
For example, if I had two candidates for a PhD position and one clearly knew ALL THE THINGS but didn't seem to necessairly fully understand them, whereas the other had a poor grasp of facts but a great grasp of basic concepts...it'd be a tough choice. 90% of the factual retention actually required for science can be replaced with "ask google", so the real trick is in knowing WHAT to ask google.


And then there's communication, which is an incredibly important skill. If someone knows a shitload but writes like a ten year old, that's not good. Writing style (and hell, even penmanship) counts, even though this is somewhat hilarious.
User avatar
Jonas
Off Topic Productions
Off Topic Productions
Posts: 14224
Joined: Sat Apr 24, 2004 9:21 pm
Location: Hafnia

Re: We should LOL @ this "no government" and "living free" g

Post by Jonas »

DDL wrote:Writing style (and hell, even penmanship) counts, even though this is somewhat hilarious.
Makes perfect sense to me. What good is your talent if you can't explain anything to anyone?
Jonas Wæver
Chief Poking Manager of TNM

I've made some videogames:
Expeditions: Rome
Expeditions: Viking
Expeditions: Conquistador
Clandestine
User avatar
gamer0004
Illuminati
Posts: 1215
Joined: Sun Mar 09, 2008 4:53 pm

Re: We should LOL @ this "no government" and "living free" g

Post by gamer0004 »

It depends on the field of course. In science (broadly defined, including economics and psychology) writing moderately well will in most cases do - although specific concepts and ideas are more likely to be noted and successful if they're appealing, and writing can contribute hugely to that.
In history and social science it is arguably more important to write well. The one who most eloquently writes down his or her thoughts is most likely to be the one whose ideas are widely quoted and accepted.
At least, that's my rather limited experience.

@DDL: the thing about knowledge and skills is their relative values depend on their definition. For example, it is very important and useful to know who is and which papers are important. Just using Google scholar can't replace knowing where a specific field "is at" and it is easy to miss important new (or even quite old) contributions. Without such knowledge it will be difficult to even know where to begin.
Unfortunately, such knowledge is also the kind of knowledge which is very difficult to test (and subject to rapid change). Textbook knowledge in most cases isn't very useful to test (it can simply be read if necessary), skill in applying the textbook concepts is more important, but even that won't get one anywhere in academics without the aforementioned knowledge of research and people in the field in general.
User avatar
Jaedar
Illuminati
Posts: 3937
Joined: Fri Mar 20, 2009 3:01 pm
Location: Terra, Sweden, Uppsala.

Re: We should LOL @ this "no government" and "living free" g

Post by Jaedar »

Jonas wrote:I'm not sure whether to be relieved or terrified that the only grading system I'm subject to anymore is measured in thousands of dollars and units shipped.

I mean I guess there's review scores as well, but they don't seem as important, in comparison...
Lot of developer bonuses seem to be based on metacritic score, but since you're indie(so far....?) that won't matter.
"Delays are temporary; mediocrity is forever."
odio ergo sum
AEmer
Illuminati
Posts: 1490
Joined: Fri Jan 26, 2007 12:04 am

Re: We should LOL @ this "no government" and "living free" g

Post by AEmer »

So my claim is the following:
We can actually study the concept of merit, and how to best assign merrit.

The current system is as follows: Universities have a reputation to uphold. If they put out crappy diplomas, their esteem will (theoretically) decrease, and people will find them less attractive. If they put out good diplomas, their esteem will (theoretically) increase. Therefore, they're incentiviced towards making a grading scheme which isn't crappy, but accurately measures peoples merit.

So that's the theory. But in actual fact, universities just need to uphold their reputation in any way they can; so long as they are well published, have charismatic professors and scientists, and have a strong media profile, the reputation would probably be pretty hard to ruin. Put on high entrance requirements, and the majority of the merit assesment is not even in anything tested at university, but comes from a cursory reading of previous grades and the hardship of attending uni at all: you can only pass if you get in, and you can only pass if you have the means to go through the years spent in education not doing anything that earns you money.

So just with these facts alone, it becomes clear that the incentives universities have to get merit right aren't that big. In fact, just being expensive and prestigious, and using some arbitrary fashion to ensure that a certain number of people always fail, combined with headhunting professors and having high entrance requirements would probably do about as well as many current universities. With this kind of sorting and a large enough battery of exams and presenting relevant subject material, the graduating students will probably be decent enough that everything will keep running smoothly, even if merit is assigned very unfairly among any given group of students.

So we have two alternatives: We can either presume that this system is as good as it gets, or we can presume that it is not as good as it gets. I presume that it is not as good as it gets because the universities who design the merit system do not have that much rational self-interest to getting it right. My argument is that it's not optimal because nobody really counts on it being optimal: So long as the job is done, the system is able to sustain itself.

I'd argue that almost regardless of how tricky it is to develop better merit assesment, doing some amount of dedicated work including actual simulations and studies and experiments would help, because with such at least _someone_ would have an incentive towards understanding the problem and getting it right.
User avatar
gamer0004
Illuminati
Posts: 1215
Joined: Sun Mar 09, 2008 4:53 pm

Re: We should LOL @ this "no government" and "living free" g

Post by gamer0004 »

AEmer wrote:So my claim is the following:
We can actually study the concept of merit, and how to best assign merrit.

The current system is as follows: Universities have a reputation to uphold. If they put out crappy diplomas, their esteem will (theoretically) decrease, and people will find them less attractive. If they put out good diplomas, their esteem will (theoretically) increase. Therefore, they're incentiviced towards making a grading scheme which isn't crappy, but accurately measures peoples merit.

So that's the theory. But in actual fact, universities just need to uphold their reputation in any way they can; so long as they are well published, have charismatic professors and scientists, and have a strong media profile, the reputation would probably be pretty hard to ruin. Put on high entrance requirements, and the majority of the merit assesment is not even in anything tested at university, but comes from a cursory reading of previous grades and the hardship of attending uni at all: you can only pass if you get in, and you can only pass if you have the means to go through the years spent in education not doing anything that earns you money.

So just with these facts alone, it becomes clear that the incentives universities have to get merit right aren't that big. In fact, just being expensive and prestigious, and using some arbitrary fashion to ensure that a certain number of people always fail, combined with headhunting professors and having high entrance requirements would probably do about as well as many current universities. With this kind of sorting and a large enough battery of exams and presenting relevant subject material, the graduating students will probably be decent enough that everything will keep running smoothly, even if merit is assigned very unfairly among any given group of students.

So we have two alternatives: We can either presume that this system is as good as it gets, or we can presume that it is not as good as it gets. I presume that it is not as good as it gets because the universities who design the merit system do not have that much rational self-interest to getting it right. My argument is that it's not optimal because nobody really counts on it being optimal: So long as the job is done, the system is able to sustain itself.

I'd argue that almost regardless of how tricky it is to develop better merit assesment, doing some amount of dedicated work including actual simulations and studies and experiments would help, because with such at least _someone_ would have an incentive towards understanding the problem and getting it right.
Interesting thought. There is yet another issue: government funding. In the Netherlands, universities get money for every diploma they hand out. If a student graduates in three years, that's good, if that student takes 6 years, that's bad, because you're spending twice as much resources and get only the same amount of funding. This obviously creates unwanted incentives, although universities are already regulated and monitored on quality and what their programmes should cover.
I think the current system (at least here in NL) is quite decent: those who pass are generally significantly more knowledgeable than those who do not. Those who score 9/10 are not necessarily better than those who score 7/10: some people are quite smart but are simply not suited for doing tests in general or timed tests or multiple choice tests &c. However, in general those who score higher grades tend to be the brighter and more knowledgeable students.
AEmer
Illuminati
Posts: 1490
Joined: Fri Jan 26, 2007 12:04 am

Re: We should LOL @ this "no government" and "living free" g

Post by AEmer »

Yeah, and look, I'm not saying that it's going to be perfect, ever, or that the current system is completely hopeless...

I'm just saying, this isn't something which should be a political decision. It should be determined by psychologists, economists, mathematicians and game theory people. Like, apply the scientific method to this bitch of a problem.
User avatar
Jaedar
Illuminati
Posts: 3937
Joined: Fri Mar 20, 2009 3:01 pm
Location: Terra, Sweden, Uppsala.

Re: We should LOL @ this "no government" and "living free" g

Post by Jaedar »

You certainly have a point aemer. However, for some reason teaching is not considered prestigeful in the slightest by most of the population.
"Delays are temporary; mediocrity is forever."
odio ergo sum
AEmer
Illuminati
Posts: 1490
Joined: Fri Jan 26, 2007 12:04 am

Re: We should LOL @ this "no government" and "living free" g

Post by AEmer »

But that's so ridiculous!

I know that food science is pretty much the bum of science, because running experiments accross the timespan to actually test out food science is complicated and hard to control for...I can accept this without much trouble because some things are just out of our reach.

But not so with merit testing. I mean, that's pretty much exactly the type of thing where you have large groups of similar people, room for controls, and where you'll have plenty of willing participants to use as subjects. You even have those UN control studies which try to figure out how well learned populations of various nations are, so you have pretty much a perfect model for figuring this stuff out.

Even though the studies will have to be long term, collect enough data and you can find out what's efficient and what isn't.

The most important thing in programming is the execution environment. That's compiler, interpreter, system, the whole shabang. All of the execution environment is self-reproducing; that is, you compile the compiler using the compiler itself. You bootstrap it. You say, if my compiler can't compile itself, it isn't good enough. This self-application is a cornerstone in computerscience as a whole - you want programmers to use their own tools, you want them to eat their own dog food.

If the scientific method should really be used to guide research and development - if academia should really be allowed to influence as much of society as it does - it should first and foremost be used to improve academia! And since academia is largely a merit based system, development of merit testing must be thought of as crucial, because it in turn will have the most influence on making academia successful. I mean, I can't prove that statement - I can only say that self application is super useful in my own field...and that the way with which merit and courses and so on are developed would absolutely not fly if it was thought of as a research project. There is no peer review; no check to ensure competence; and there are no overall guidelines within each subject which have been proven to be efficient. It's all just a bunch of (fairly experienced) individuals winging it.
DDL
Traditional Evil Scientist
Traditional Evil Scientist
Posts: 3791
Joined: Mon Oct 17, 2005 10:03 am

Re: We should LOL @ this "no government" and "living free" g

Post by DDL »

Not sure I wholly agree. I think there's certainly scope for improvement, because it's a system run by people (and, as I've said time and time again, people are dicks)...but in a sense universities DO act like your self-compiling compiler: at least in the sciences (I'm not going to pretend I know anything about the arts side) a substantial percentage of students graduating are going to be funneled back into PhD programs, so if you haven't got a good system in place for actually picking out the good ones, you're going to end up with a whole lot of shitty PhD students.

Plus of course the grading system doesn't usually have to be uniform, since you're not looking at a normally distributed sample set: you don't need a smooth range of grades from useless to awesome, because you're already looking at people good enough to study the course at degree level. You can pretty much reduce those down to

"can't cut it/doesn't give a shit/wtf is this person doing here" <-- the 'fail' grade, should rarely happen

"can make the grade with effort"

"can make the grade easily"

"is exceptional" <-- if they're good enough to reach/exceed this threshold, they're good enough for pretty much everything, so you don't need additional benchmarks

which in percentage terms means you're actually looking at a smallish window between maybe 50 and 70%, with anything falling outside those boundaries being "why are you here?" and "holy shit when can you start?" respectively.
User avatar
Jonas
Off Topic Productions
Off Topic Productions
Posts: 14224
Joined: Sat Apr 24, 2004 9:21 pm
Location: Hafnia

Re: We should LOL @ this "no government" and "living free" g

Post by Jonas »

It works pretty well in humanities from my experience. The only real problem is different people have different ideas of what should be expected of the students, but the error margin isn't huge, it typically boils down to whether somebody gets 10 or 12, both of which are rather good grades. The only really important threshold is between fail and pass, and it's not my impression that there's ever much doubt at that end of the scale.

I'm not sure how this topic ended up being a discussion of grading systems, but I'm pretty happy with that.
Jonas Wæver
Chief Poking Manager of TNM

I've made some videogames:
Expeditions: Rome
Expeditions: Viking
Expeditions: Conquistador
Clandestine
User avatar
Jaedar
Illuminati
Posts: 3937
Joined: Fri Mar 20, 2009 3:01 pm
Location: Terra, Sweden, Uppsala.

Re: We should LOL @ this "no government" and "living free" g

Post by Jaedar »

Jonas wrote: I'm not sure how this topic ended up being a discussion of grading systems, but I'm pretty happy with that.
The tides of offtopica are mysterious indeed. Anyway, grades are cool and all but I'd rather see improvements on the actual teaching side. Of course, its really hard to measure those improvements without good tests so I guess the two are inextricably linked.
"Delays are temporary; mediocrity is forever."
odio ergo sum
Post Reply