LiangDong

Experiment Design

Experiment Design

#Con­fer­ence HCI In­ter­na­tional

ICCM

Demos

Experiment Design

#Graph Pop­ular­ity

  1. New York Times: Man­u­fac­tur­ing costs rose from 2004 to 2014 in most of the top 25 ex­port­ing coun­tries

  2. Git­hub Git

  3. Busi­ness In­sider: Apple no longer dom­in­ates the tab­let mar­ket it cre­ated

  4. Min­imum wage since 1938

Questions

Point read­ing: Is A lar­ger than cer­tain amout in one year

Com­par­ison: Is A lar­ger than B

Trend Ques­tion: The trend of A over time

Which plot is bet­ter in cer­tain cir­cum­stance?

How do eye move­ment pat­terns(cog strategy) de­pend on the data?

Assumption

If the par­ti­cipant spends more time on fix­a­tion of one ob­ject, he spends more time un­der­stand­ing it.

Goal

  1. In­vest­ig­ate the cong­nit­ive pro­cess of read­ing graphs on the in­ter­net

  2. Eval­u­ate the be­ne­fits and short­comes of dif­fer­ent graphs based on cog­nit­ive find­ings

Validate the opinions or not?

  1. As­so­ci­ate the le­gend -> Find di­men­sion -> Find as­so­ci­ated data­point -> Get data­point value

  2. Line graph rep­res­ents the trend fairly well, with grouped charts it can be dif­fi­cult to tell the dif­fer­ence between totals in each group however it’s ideal to com­pare between each ele­ment in cat­egor­ies.

Participants

Results

Line Graph point reading

GitHub Logo

Per­son 1:

(1) Le­gend -> X axis -> Line -> Y axis -> Line -> X axis

(2) Le­gend -> X axis -> Line -> Y axis -> Line -> X axis -> Le­gend -> Y axis -> Line

(3) Le­gend -> X axis -> Line -> Le­gend -> Y axis

Per­son 2:

(1) X axis -> Le­gend -> Line -> X axis -> Line -> Y axis

(2) X axis -> Le­gend -> Line -> Y axis -> X axis -> Line -> Y axis

(3) X axis -> Le­gend -> Line -> X axis -> Line -> Y axis -> X axis

Per­son 3:

(1) Le­gend -> X axis -> Line -> Y axis

(2) not clear

(3) Le­gend -> X axis -> Line -> Y axis -> Line

Bar Graph point reading

Per­son 1:

(1) Le­gend -> X axis -> Bar -> Le­gend -> Bar -> Y axis

(2) Le­gend -> X axis -> Le­gend -> Bar -> Y axis

(3) Le­gend -> X axis -> Le­gend -> Bar -> Y axis

Per­son 2:

(1) Le­gend -> X axis -> Bar -> Y axis -> Bar -> Y axis -> Bar -> Le­gend -> Bar -> Y axis

(2) Le­gend -> X axis -> Bar -> Y axis -> Bar -> Y axis -> Bar

(3) X axis -> Le­gend -> Bar -> X axis -> Bar -> Y axis -> Bar -> X axis -> Bar -> Le­gend -> Bar

Per­son 3:

(1) not clear

(2) Le­gend -> X axis -> Bar , not clear

(3) Le­gend -> X axis -> Y axis , not clear

Discoveries

starting gaze point

For per­son 1: start­ing gaze point is al­ways le­gend: 20/​20

For per­son 2: start­ing gaze point is 10/​20, ex­actly half.

For per­son 3: start­ing gaze point is un­clear be­cause the ex­per­i­ment data is not re­li­able.

Software Tools

Eye Move­ment Clas­si­fic­a­tion Soft­ware From Oleg Ko­mogort­sev

jetty Web­socket

d3

Boot­strap

Result

ACT-R

Line graph

Point read­ing: Is A lar­ger than cer­tain amout in one year

3.095S, 3.580S, 4.015S

Com­par­ison: Is A lar­ger than B

3.015

Trend Ques­tion: The trend of A over time

3.670

Bar graph

Line graph

Point read­ing: Is A lar­ger than cer­tain amout in one year

3.095S,

Com­par­ison: Is A lar­ger than B

3.015

Trend Ques­tion: The trend of A over time

3.670, 3.385

#per­son 1

1 2.3 ——————– 2 4.23333333333 ——————– 3 4.03333333333 ——————– 4 4.0 ——————– 5 5.6 ——————– 6 3.8 ——————– 7 4.26666666667 ——————– 8 3.23333333333 ——————– 9 6.53333333333 ——————– 10 3.3 ——————– 11 4.7 ——————– 12 5.4 ——————– 13 5.96666666667 ——————– 14 5.93333333333 ——————– 15 5.23333333333 ——————– 16 4.0 ——————– 17 3.43333333333 ——————– 18 3.33333333333 ——————– 19 4.36666666667 ——————– 20 5.1 ——————–

#per­son 2

1 10.7 ——————– 2 10.2333333333 ——————– 3 4.93333333333 ——————– 4 7.56666666667 ——————– 5 6.96666666667 ——————– 6 5.2 ——————– 7 4.66666666667 ——————– 8 6.1 ——————– 9 4.33333333333 ——————– 10 4.26666666667 ——————– 11 9.46666666667 ——————– 12 6.36666666667 ——————– 13 6.56666666667 ——————– 14 6.63333333333 ——————– 15 4.8 ——————– 16 20.2666666667 ——————– 17 6.96666666667 ——————– 18 4.66666666667 ——————– 19 3.4 ——————– 20 4.76666666667 ——————–

#per­son 3

1 7.5 ——————– 2 16.5 ——————– 3 4.06666666667 ——————– 4 4.33333333333 ——————– 5 8.0 ——————– 6 5.96666666667 ——————– 7 5.3 ——————– 8 5.36666666667 ——————– 9 9.03333333333 ——————– 10 5.06666666667 ——————– 11 5.46666666667 ——————– 12 6.06666666667 ——————– 13 4.36666666667 ——————– 14 6.53333333333 ——————– 15 4.83333333333 ——————– 16 4.7 ——————– 17 5.66666666667 ——————– 18 7.1 ——————– 19 3.9 ——————– 20 16.4666666667 ——————–

25 Jul 2015