[opeverkosto] Koli Calling: Workshop of CS education Tools

Koli Calling: Workshop of CS education Tools
Koli, Finland, November 13-16th, 2008

Call for Tools

A strength of computing education research is its
focus on practice. One important subfield within
this area deals with developing various kinds of
software tools to assist students' and/or
teachers' work. Such tools include, for example,
automatic assessment, visualization, simulation
and course management tools, as well as dedicated
learning environments or learning objects. They
have had a considerable effect on teaching
practices and students' learning outcome in many
institutes, and many successful experiences have
been reported. Typical research papers related to
this software include system papers, in which the
rationale for the new tool is described in
addition to presenting its functionality, and
evaluation papers, where the focus is on
evaluating the impact of the tool on students'
learning process, learning outcomes or teacher's
work. Many papers include both of these
elements, though.

In many cases developing this kind of software
requires a major effort from the researcher(s)
and may take from several months to several years
or full time work. Unfortunately, this work is
rarely given proper credit in scientific
evaluations as typically research papers are
considered the only important merit. This
practice can well be supported by a claim that
from the research point of view, the software is
only a tool required to carry out the experiment,
or test the feasibility of the novel idea - in a
similar way as physical gadgets are needed in
experimental physics research. Moreover, the
software itself has no scientific peer review
system to assess its quality. Even though there
are some resource repositories, like SIGCSE
Education links, these have not been very
successful. For example, SIGCSE education links
has currently in total only 103 entries for the
whole CS field, of which only 21 are classified
as courseware. Moreover, currently no new
submissions are accepted. Another type of
evaluation process is applied in Courseware
competitions, such as Premier Award of
Engineering Education Courseware, but very few
software get credit through these kinds of
procedures.

A significant drawback of this state of affairs
is that there is not much incentive for
developing the software to match something more
than the needs of the developer's course / class
/ institute. When the initial contribution has
been published in papers, it is quite difficult
to get credit for further development work like
improving usability, generalizing the available
features to allow easier adoption of the tool,
writing high quality tutorials and manuals, and
extending the tool to cover new topics.
Consequently very few tools have achieved a wide
international dissemination - most are used only
in their developers' home institutes or have been
forgotten totally when the initial author has
left the institute, and maintenance is no longer
available.

We argue that a new procedure is needed to
evaluate the quality of software tools to give
them proper scientific credit. With a rigorous
evaluation process high quality educational
software should gain a merit comparable to
research papers. Moreover, such software has much
better chances for wider dissemination, which
would be beneficial for the whole CS education
field.

With this rationale, we present a CALL FOR TOOLS
for computing education. This workshop on
computing education tools will be organized as a
part of the Koli Calling 2008 conference. The
workshop will include presentations of accepted
tools, discussions on relevant directions of
future tools research and appropriate actions to
promote research, development, and dissemination
of educational tools / software for computing
education.

Original submissions in all areas related
computing education are welcome. Topics of
interest include, but are not limited to:

     * Programming support tools
     * Visualization tools
     * Simulation tools
     * Automatic analysis / assessment tools
     * Course management tools
     * Learning environments

A submission should be an original novel tool for
some purpose, or a new version of an existing
tool with a clearly identifiable new
contribution. The submission should include

    1. A short paper (3-5 pages in ACM double
column format) that summarizes the rationale of
the tool
           * What is the problem in CS education
the tool addresses and how does it address it?
           * What are the arguments / evidence
that the tool works effectively to meet this aim?
           * A brief description of the main
functionalities/features provided by the tool.
           * In which way does the tool differ
from other tools solving the same or similar
problem? What are the key contributions of the
tool?
           * Summary of possible publications
about the tool and its evaluation results.
    2. Instructions to access a web site that includes
          1. A tutorial or guided tour of using the tool
          2. Relevant documentation of the tool
          3. The tool itself
          4. Source code of the tool (if
appropriate) and instructions to install the tool
on a new platform
          5. Any other relevant resources
    3. If the tool is only accessible on a server
and requests logging in, the submission must
include relevant information to allow reviewers
to access the tool. The accounts should
preferably be anonymous.

Note that the part two of the submission is more
important from the review process point of view
than the summary paper (part one).

The submissions will be evaluated based on the following criteria:

     * Originality compared with other similar
tools or tools addressing a similar problem
     * Argumentation and evidence to support the claimed contribution
     * Potential or demonstrated impact on further
research, such as supporting novel concepts or
working methods
     * Potential or demonstrated impact on
improving teaching practice or learning outcomes
     * Generality to address the task the tool
handles (for example, how narrow or wide set of
algorithms a visualization tool can present, or
how well the tool is customizable for the needs
of a new target institution)
     * Usability
     * Documentation quality
     * Software quality
     * Maintenance support
     * Possible additional resources

Note that a contribution cannot be based on
unpublished or proprietary code that is
inaccessible. Building the contribution on a
commercially available resource is not an
obstacle, however, as the main criteria is
accessibility. Yet, for the evaluation process,
access to all relevant resources should be free.
In addition, the libraries and other resources
needed to develop the system further must be
obtainable for all researchers.

Koli Calling Educational Tool Award

The workshop program committee will select one of
the accepted tools to receive the Koli Calling
Educational Tool Award at the conference.

Important dates

Submissions due, September 1st 2008
Notification of acceptance, October 2nd
Registration deadline, October 30th
Workshop, November 13th (evening) - 16th (lunch-time)

The tool papers (part 1 in the submission) will
also be published in the proceedings of the Koli
Calling conference, after the conference.

Program committee

Ari Korhonen, Helsinki University of Technology, Finland (chair)
Mordechai (Moti) Ben-Ari, Weizmann Institute of Science, Israel
Pierluigi Crescenzi, Università degli Studi di Firenze, Italy
Mike Joy, University of Warwick, UK
Mikko-Jussi Laakso, University of Turku, Finland
Raymond Lister, University of Technology, Sydney, Australia
Lauri Malmi, Helsinki University of Technology, Finland
Arnold Pears, Uppsala University, Sweden
Guide Roessling, Darmstadt University of Technology, Germany
Tapio Salakoski, University of Turku, Finland
Carsten Schulte, Freie Universität Berlin, Germany

For more information, see conference web site
http://www.cs.joensuu.fi/kolistelut/
or contact Ari Korhonen (archie_at_cs.hut.fi) or Lauri Malmi (lma_at_cs.hut.fi)

-- 
----------------------------------------------------------------------
Lauri Malmi, professor
Helsinki University of Technology,
Department of Computer Science and Engineering,
P.O.Box 5400, 02015 HUT
email:  Lauri.Malmi_at_hut.fi
puh. +358 - 9 - 451 3236, fax +358 - 9 - 451 3293
----------------------------------------------------------------------