Home
  • Deutsch (DE-CH-AT)
  • English (United Kingdom)
  • Japanese(JP)
SML@b Agent Measurement PDF Print

ban_collage

The following approach of Software Agent Measurement Framework (SAMF) was developed by Prof. Dr. Cornelius Wille (see Wille: Software Agent Measurement Framework, Shaker Publ., 2005 and Dumke/Mencke/Wille: Quality Assurance of Agent-Based and Self-Managed Systems, CRC Publ., Boca Raton, 2010). This approach is a general concept and was implemented in some prototypes (see the references above).The basic idea consists of measurement of agents, agents cooperations and agent-based systems.

samf1a samf1b samf1c

 

Here we will given only the list of metrics and their appropriate empirical evaluations as following:

Software Agent Measurement

The metrics for the agent design level are:

  • Software agent size : The size considers both aspects of an agent: the functional size and the physical size of a software agent.
  • Software agent component structure : The structure depends on the kind of the agent (intelligent, reactive, deliberative etc.), and the agent interface is related to the kind of agent coupling (as fixed, variable or evolutionary).
  • Software agent complexity : The complexity is divided into the computational and psychological complexity and should be measured using both concrete aspects.
  • Software agent functionality : This aspect considers the appropriateness of the agent with respect to the requirements.

The metrics for the agent description level are:

  • Software agent development description level : It considers the completeness of the development documentation (including tests and change supports).
  • Software agent application description level : The metric includes the quality (readability, completeness, on-line support etc.) of the user documentation.
  • Software agent publication description level : This metric considers the public relations for using the software agent and involves the system description.

The metrics for the agent working level are:

  • Software agent communication level : Considers of the size of communication and the level of the conversation required to sustain the activities.
  • Software agent interaction level : This metric is related to the agent context and environment and their different kinds of actions (as transformation, reflecting, executing, modification, commands, perception, deliberation).
  • Software agent learning level : This metric evaluates the skills, intentions, and actions of extending the agent facilities itself.
  • Software agent adaptation level : The adaptation metric considers facilities of an agent changing in order to react to new conditions in the environment.
  • Software agent negotiation level : The measuring is based on the evaluation of facilities like the agent intentions, conflict resolution, and realized commitments for successful negotiation.
  • Software agent collaboration level : This metric is oriented towards the agent’s facility to work together with other agents.
  • Software agent coordination level : The agent’s facility of managing any one agent task is considered.
  • Software agent cooperation level : This metric considers the volume and efficiency of an agent relating to a common task.
  • Software agent self-reproduction level : The number of destroyed agents related to repaired agents is counted.
  • Software agent performance level : This metrics considers the task related performance of an agent.
  • Software agent specialization level : The metric considers the degree of specialization and the degree of redundancy of an agent.

Note that the metrics-based analysis of the agent behavior is one of the new and extended areas in software measurement of agent-based systems.

The empirical characteristics of the agent design level metrics:

  • A large agent size can cause a low performance and mobility.
  • The structure does affect the changeability.
  • A high computational complexity leads to weak performance.
  • A high functionality can inhabit the chosen object-oriented implementation paradigm.

The empirical characteristics of the agent description level metrics:

  • The description level determines the maintainability of an agent.
  • This evaluation considers the usability of an software agent.
  • A high publication level supports spreading of the agent use.

The empirical characteristics of the agent working level metrics:

  • A high communication intensity can affect a flexible application.
  • This aspects expresses the activity of an agent.
  • This level is based on the type of agent and its roles in the system.
  • The facility of adaptation determines the stability of the agent implementation.
  • This level determines the success of an agent activity relating to common tasks.
  • The collaboration of an agent classifies its roles in the given tasks.
  • This level determines the role of the agent in an administration hierarchy.
  • This level determines the effectiveness of common tasks realization.
  • This level determines the stability of an software agent itself.
  • A high agent performance is related to all kinds of agent activities.
  • A high specialization can lead to a high performance.

 

Software MAS Measurement

The metrics for the MAS design level:

  • Agent system size : The measured system size includes the potential number of (active) agents and their contents; further, the size is related to the environment.
  • Agent system functionality : This metric considers the realization of all of the functional system requirements.
  • Agent system component structure : This metric includes agent the type of organizational structure (hierarchies or egalitarian), the degree of parallelism, the kinds of organizational functions (representational, organizational, conative, interactional, productive, or preservative).
  • Agent system complexity : One of these measured aspects leads to the degree of the organizational dimensions (social, relational, physical, environmental, and personal).

The metrics for the MAS description level:

  • Agent system development description level : This metric considers the integration of the agent concepts and dynamics and their sufficient documentation.
  • Agent system application description level : This considers the user documentation of all aspects of the system applications related to the different user categories.
  • Agent system publication description level : Publication metrics evaluate the user acceptance and marketing aspects of the agent-based system application.

The metrics for the MAS working level:

  • Agent system communication level : The number of ACLs between the different kinds of software agents and their different roles and actions.
  • Agent system interaction level : This metric deals with the average types of interactions relating to the agents and their roles in the environment of the agent-based system.
  • Agent system knowledge level : This metric measures the results of agent learning for agent-based system (based on the different kinds of agents, either tropistic or hysteretic).
  • Agent system lifeness level : This metric is based on the agent adaptation which reflects the adaptation level of the whole agent-based system.
  • Agent system conflict management level : The system success is based on agent negotiation and considers the relations between the different kinds of a fair play in the realization of the system tasks.
  • Agent system community level : This metric considers the level of different agent communities based on the agent collaboration.
  • Agent system management level : This system metric is based on the agent coordination level with respect to the whole agent system structure.
  • Agent system application level: This metric is related to the application area and the different agent roles in cooperation.
  • Agent system stability level : The stability measure is based on the agent self-reproduction.
  • Agent system performance level : The handling with object to realize special tasks through the different agents is considered.
  • Agent system organization level : The different agent roles (archivist, customer, mediator, planner, decision-maker, observer, and communicator) are considered.

The empirical characteristics of the MAS design level:

  • A small agent system size can reduce the application area.
  • The system structure relates to the performance and the system changeability.
  • This aspect influences the system applicability.
  • The distribution of the functionality in the system components influences their flexibility.

The empirical characteristics of the MAS description level:

  • The system description effects of system maintenance.
  • A good application description is a precondition for efficient use of the whole system.
  • A good system publication supports the spreading of the system especially within the educational area.

The empirical characteristics of the MAS working level:

  • This level characterizes the intensity of the conversations and is describes the agent collaboration.
  • Many interactions are based on high cooperation.
  • This aspect determines the knowledge-based foundation of the agent-based system.
  • This aspects is based on the adaptability of the agents and characterizes the system maintenance effort.
  • A high conflict management level leads to a high system stability.
  • A high community level is caused on collaboration for different classes of system application.
  • A high management level expresses a good agent organization level.
  • A high application level is reflected in an effective task-oriented agent cooperation.
  • A high stability level includes the agent self-reproduction and other system error handling facilities.
  • This level includes the agent performance and the performance of the environment.
  • This level leads to an efficient distribution of the agent roles and their administration.

 

Software Agent and MAS Development Measurement

The metrics for the agent development life cycle:

  • Software agent phases level : The characteristics (size, structure, complexity) in the different development phases are considered.
  • Software agent milestones level : This metric evaluates agent development with respect to a milestone.
  • Agent requirements workflow level : This metric considers the implemented requirements during the development phases.

The metrics for the agent development method level:

  • Software agent methodology level : The level of the development method used is quantified.
  • Software agent paradigm level : This metric evaluates the appropriateness of the chosen development paradigm.
  • Software agent CASE level : This metric quantifies the tool support for the agent implementation.

The metrics for the agent development management level:

  • Agent project management level : This set of metrics considers the management level of the development risks and methods.
  • Agent configuration management level : This considers the successful of the version control with respect to an agent.
  • Agent quality management level : This set of metrics considers the quality assurance techniques related to an agent.

Now, we consider the development process of the MAS and define the following appropriate software metrics for the measurement and evaluation of these aspects.

The metrics for the MAS development life cycle:

  • Agent system phases level : This evaluation considers the system metrics of size, structure and complexity during system development.
  • Agent system milestones level : The metric evaluates MAS development with respect to a milestone.
  • System requirements workflow level : The requirements implementation during the development phases in the whole system is considered.

The metrics for the MAS development method:

  • Software agent methodology level : The level of the development method used is quantified
  • Software agent paradigm level : This metric evaluates the appropriateness of the chosen development paradigm
  • Software agent CASE level : This metric quantifies the tool support for the agent implementation

The metrics for the MAS development management level:

  • System project management level : The management level of the development risks and methods of the system is considered.
  • System configuration management level : This metrics includes the evaluation of the dynamic aspects of the system configuration.
  • System quality management level : The quality assurance techniques related to the whole agent-based system is considered.

The metrics for the agent developer level:

  • Agent developer skill level : This metric is related to the skills to develop and implement an software agent.
  • Agent developer communication level : The ability of the developer to improve his work by collaboration and cooperation is considered.
  • Agent developer productivity level : This metric evaluates the quantity of work.

The metrics for the agent software resources level:

  • Agent software paradigm level : This metric evaluates the appropriateness of the chosen software basis and used software components for the implementation of an software agent.
  • Agent software performance level : This metric addresses the software components and their effectiveness.
  • Agent software replacement level : This metric considers the effort of adaptation when using different versions of the basic software.

The metrics for the agent hardware resources level:

  • Agent hardware reliability level : This metrics considers the reliability of the types of hardware required for running the software agent.
  • Agent hardware performance level : This set of metrics considers the platforms used for an agent.
  • Agent hardware availability level : The average availability of the different platforms used from a (mobile) agent is considered.

The metrics for the MAS developer level:

  • System developer skill level : This metric is based on the agent developer skills and is extended by the (dynamic) system characteristics.
  • System developer communication level : This set of metrics considers the ability of the developer(s) to improve his work by collaboration and cooperation.
  • System developer productivity level : The quantity of work is considered.

The metrics for the MAS software resources level:

  • System software paradigm level : The appropriateness of the chosen software basis and COTS system used for the implementation of the agent-based system is evaluated.
  • System software performance level : This metric considers the evaluation of the efficiency of the involved software base and the external components.
  • System software replacement level : The adaptation to the different versions of the basic software is considered.

The metrics for the MAS hardware resources level:

  • System hardware reliability level : The reliability of the kinds of hardware for running the agent-based system is considered.
  • System hardware performance level : This set of metrics considers the platforms used for an agent-based system.
  • System hardware availability level : The average availability of the different platforms used with the agent-based system is considered.

The empirical characteristics of the agent development life cycle:

  • A high phase level is expressed by a high level of verification.
  • This level expresses the correct timing of the agent development.
  • This level is caused by the timely realization of the requirements for the agent implementation.

The empirical characteristics of the agent development method level:

  • This level means that the development method should be adequate for the type of agent implementation.
  • A high paradigm level is caused by the appropriate choice for the implementation technique.
  • This level reflects the tool support during agent development.

The empirical characteristics of the agent development management level:

  • A high level of management is involved in the system project management.
  • This level reflects the quality of version control for the software agent.
  • This level reflects the quality assurance techniques related to the agent development.

The empirical characteristics of the MAS development life cycle:

  • This level is caused by appropriate development results for an efficient system realization.
  • This level is related to all development aspects in the planning phase of their realization.
  • A high workflow level evaluates the appropriateness of the realized system requirements.

The empirical characteristics of the MAS development method level:

  • A high methodology level reflects the use of appropriate development techniques.
  • This level determines the appropriateness of the chosen techniques for the system implementation.
  • This level includes the set of different tools in order to support system development.

The empirical characteristics of the MAS development management level:

  • This level describes the timing and the appropriate use of resources for system development.
  • This  level is effected by using version control for all parts of the agent-based system.
  • This level includes the different quality assurance techniques.

The empirical characteristics of the agent developer level:

  • A high skill level expresses a good developer specialization for agent implementation.
  • The communication is an indicator for the efficient resolution of any questions.
  • A high productivity includes as well as functionality and the quality of the software agent.

The empirical characteristics of the agent software resources level:

  • This level reflects the appropriateness of the chosen paradigm.
  • This level is a precondition for the agent performance itself and is related to the system software used.
  • A high replacement level reflects a large quantity of agent maintenance and migration.

The empirical characteristics of the agent hardware resources level:

  • This level reflects the different platforms which will be used by a mobile agent.
  • This level considers also the potential types of platforms.
  • A high availability level is a precondition for the mobility of an agent.

The empirical characteristics of the MAS developer level:

  • This level reflects the different kinds of knowledge used to develop the different components of the agent-based system.
  • A high communication level is based on the successful design techniques of the participants.
  • This level is related to the development of the different system components.

The empirical characteristics of the MAS software resources level:

  • This level is divided into the different system components.
  • The system performance of the COTS used indicates a high performance level.
  • This level is divided into the evaluation of the different components of the agent-based system.

The empirical characteristics of the MAS hardware resources level:

  • This level includes all platforms of the implemented environment of the agent-based system.
  • This level is a basis for the efficiency of the agent-based system.
  • This level expresses the stability of the system used.

This Set of AOSE Metrics spans a measurement space in a holistic manner and shows the extentions to the other/before paradigms such as OOSE, CBSE and SOSE. Applications are given at following:

 

Software Aglets Performance Agent Academy Evaluation JDMK-Based Infrastrucuture Analysis
samf01
samf03
samf02

 

kugel

Last Updated on Friday, 18 January 2013 15:45