Virtual Worlds using Cell phones
Virtual world simulators like Second Life represent the most recent and most successful frontier of online services for entertainment and business. Individuals have virtual lives during these worlds using personal avatars. Bridging actual life and these virtual worlds together is challenging, but enables new application scenarios of these systems.

Existing work concentrates on the representation of objects inside virtual worlds. We feel that the overall experience is going to be improved as we can reproduce not just inanimate and passive objects like buildings and beer steins but the subjects during these virtual environments. To be able to have a seamless integration between your physical and virtual worlds, and to enhance the scale of adoption, the machine should depend on devices that individuals already use every single day.
For this reason, we feel that cell phones are a natural and ideal conduit. Within this paper, we reason that the sensors baked into commercial cell phones can be used to infer real-world activities that consequently can be reproduced in virtual settings. We discuss the difficulties related to the implementation of these a system both from algorithmic and systematic perspectives. We present the appearance of a prototype integrating a platform for inferring activities using cell phones and Second Life. Towards the best of our knowledge this is actually the first time that sensors utilized on everyday cell phones reflecting a person’s activity within the physical world continues to be expressed in virtual worlds for example Second Life.
Virtual world simulators represent the newest and most successful frontiers of online entertainment. Included in this, Second Life is probably the most popular Internet based virtual worlds when it comes to subscribers with more than 500,000 active users. A customer program known as the Second Life Viewer allows its users (called residents) to have interaction with each other through mobile avatars. Within the game, residents can socialize and take part in individual and group activities. An avatar’s appearance might be customized across an array of physical attributes. Avatars might be completely invented, with no relation to the physical part of the human user, or can be created to bear much resemblance to the human whom they represent.
However, in the present system, there’s still a divide between your virtual world and also the real one. Specifically, a person may be able to recreate his appearance but he is able to not automatically mirror/translate his current actions and movements in the real life towards the virtual lifetime of his avatar. Our vision would be to bridge the present divide between real worlds and virtual worlds, with real world activities (e.g., sitting, walking, running, cooking) along with other aspects of a person’s represented instantly in the virtual world. Our aim would be to go one step further than simply reproducing spaces (e.g., rooms, buildings, public spaces [7, 11]) and also the objects that humans interact. We focus instead on providing an online representation of humans, their surroundings as sensed through the humans themselves, and interactions using the members of their internet sites (e.g., fetched from external sources for example Facebook ).
You will find existing virtual worlds that map towards the real one with great accuracy when it comes to geographical locations and native features. Complex virtual representations of real geographical places exist like the ones supplied by Google Earth and Microsoft Virtual Earth . In the search engines Earth you’ll be able to insert 3-D representations of buildings and artefacts by way of Google Sketch. We feel that the next thing will be representing real life activities within this kind of virtual setting whereby personal avatars can move inside these artificial spaces following a real positions and activities of individuals.
Architecture from the system bridging CenceMe and 2nd Life Sensors happen to be used to support presence in virtual environments because the first experiments around the Virtual Reality Theatres . In many of these systems, individuals carry special wearable devices like helmets and wristbands with embedded sensors measuring their movements. They are special-purpose external devices accustomed to map user’s actions. Recently, the integration of sensors and 2nd Life is discussed like a future operate in , where the authors offer reproduce the whole floor of the building using data collected by way of the Plug platform. However, the main focus of is around the reproduction of the working space and never on the activities of individuals moving within it (i.e, it’s focused on places and objects and never on subjects).
Moreover, is with different non-standard hardware platform manufactured in small quantities and never available to everyone. Starting from this technological scenario, the contribution of the paper could be summarized the following: _ We discuss the systems challenges and also the design issues in integrating real life people-centric sensor systems and virtual worlds; _ We present something that is in a position to reproduce human activities and never only inanimate objects inside a virtual world the very first time; _ We discuss the appearance of a preliminary prototype in line with the integration of the system for activity inference according to mobile phones and 2nd Life.
In Section 2 we produce an overview of the primary challenges in implementing systems supporting real activity inference algorithms for virtual worlds according to data extracted using cell phones. In Section 3 we discuss our ongoing focus on the CenceMe platform that’s the basis for the practical implementation from the proposed system, and offer preliminary results which are at the root from the proposed architecture. Section 4 concludes the paper, summarizing the contribution in our work.
System Design Challenges
Within this section, we discuss the important thing issues in detecting real world activities by way of commercial cell phones and seamlessly integrating these details into virtual world’s environments.
Activity Recognition and Visualization
Activity inference, mapping and visualization are key facets of the proposed system. Activity inference includes the extraction of high-level activities (e.g., sitting, running, walking or talking) from time number of raw data sampled from sensors (e.g., accelerometer, microphone). There’s significant curiosity about the research community in designing high accuracy activity recognition algorithms [6, 12]. Many user-specified visualization policies for inferred activity may exist. Quite a few users might want to map their activities in the real world into different virtual ones. For instance, a user might want to specify the action of running corresponds to flying within the virtual world. Another interesting aspect is information accuracy.
Quite a few users may would rather discard the data if the inference isn’t sufficiently accurate given predefined quantitative thresholds. Alternatively, a default activity (e.g., standing) could be preset, which is visualized when the system is unable to perform activity recognition having a sufficient accuracy. This method can be in line with the definition of specific classifier thresholds around the Receiver Operating Characteristic (ROC) curves .
Generally, the precision of these measurements relates to the limitations of now available embedded sensors. We feel activity recognition accuracy is only going to improve later on when stronger sensors become readily available for use with increased effective inference algorithms. We consider the chance of information flow from virtual worlds towards the phones.
Of particular interest rates are researching the very best methods for reporting and representing in real life changes and interactions that exist in the virtual world. One particular very simple interaction may be the opening of the internet chat window if avatars begin a conversation in Second Life. More complicated interaction dynamics could be envisaged and also the improvements from the user interfaces from the phones will probably open a new study opportunities.
Intermittent Connectivity
Mobile phones may experience intermittent connectivity because of limits in network coverage. Virtual avatars ought to be set to some meaningful state (like the latest known state having a question mark appearing over the avatar) during intervals when no new updates are received. A far more complex solution may involve using prediction strategies to determine the present state and also the location because of the sequence from the previous states.
Delay tolerant mechanisms may also be adopted to temporarily store user status locally on the telephone in case of disconnection. Time series of state data can later be input towards the virtual world. Scientific studies are needed to figure out how to best (if) represent this type of delayed burst of physical state information within the virtual world. We feel that strict real-time action rendering very can be not necessary for most users. However, it is really an interesting aspect that needs to be analyzed at length for different activities and classes of users by way of experiments and surveys.
Scalability
Because of the potentially large number of users, designing for scalability is essential. To reduce the amount of status updates flowing in to the Second Life visualization, the status/activity from the virtual avatar is updated only if the inferred status/activity from the human user changes. This, consequently, is dependent how often the activity classifier is executed. By leveraging historical activity traces, we offer set the classifier execution frequency proportional as far as of variability from the users past behaviour. A sensor data push model in the mobile phone towards the server can be used.
We believe the cell phone is best positioned to handle the trade-off between limited local storage, data upload opportunity, and user expectations of application responsiveness. Therefore, our default approach would be to run activity recognition algorithms around the mobile device to lessen communication costs also to reduce the computational burden around the server. However, we notice that some classification algorithms might be too computationally intensive for handheld devices and offer run a portion (possibly all) from the classifier on a tailgate end server in these instances.
This may be particularly appropriate throughout the training phase of the classification model, or when data sourced from multiple users support feature vectors input towards the classifier. Low-fidelity classifiers requiring fewer resources may be used on cell phones in lack of connectivity in order to reduce transmission costs.
Data Sharing and Group Activities
The main focus of existing focus on sensor-based human activity inference is around the analysis and processing of knowledge collected by devices carried by or mounted on a single individual [6, 12]. This inference process exploits just the information extracted through the local sensors. We feel that the system should leverage the sharing of raw or processed data among differing people in physical proximity by way of short range radio communication.
This mechanism could also allow for a far more accurate inference of group activities for example talking or moving together, improving their rendering within the virtual space . Moreover, by way of this mechanism, individuals can update not just their status, but additionally those of their neighbours.
External Sensing Devices
Along with sourcing data from phones, the game recognition process might be augmented by way of external devices attached to the phones with the Bluetooth radio. These units may be designed with additional sensors to be able to improve the accuracy of activity classifiers running on the telephone, or to extend the group of the actions that may be inferred. Using a small size sensing module external towards the phone (e.g., connected to the ankle) enables more accurate activity inference (e.g., running) in contrast to using phone-embedded sensors which can be ill-positioned (e.g., within the pocket).
For this function, we created a badge-sized device, the BlueCel, integrating some various sensors including accelerometer, light, temperature and Galvanic Skin Response (GSR) sensors. The GSR sensor may be used to infer emotions like stress level. Within our vision, the machine should not depend on these devices, but on off-the-shelf platforms (i.e., the sensors integrated within the mobile phones). Simultaneously, it should be able to utilize them in a seamless and opportunistic manner when available.
Context Reflection
Virtual worlds are a great medium to represent environmental data. There’s a growing curiosity about displaying sensor data using common and popular media for example Web based portals (e.g., the Sensor Map project , the BikeNet system ). People-centric sensing is a perfect way of collecting data of these systems in real-time.
Data could be collected and displayed in virtual worlds for example Second Life inside a powerful and captivating manner by changing the look of an environment based on the sensed data. For instance, high humidity may trigger the visualization of rain or misty conditions. A pre-processing from the raw environmental data might be performed within the back-end. Users could also actively take part in the data collection process with the acquisition of multimedia content for example recordings of sounds and voices, pics and vids to be integrated within the virtual worlds. While these aspects are extremely promising, they’re out of the scope of the work.
Privacy and Social Implications
To handle their exposure, users will be able to disconnect in the virtual worlds anytime or be in a position to interrupt the sampling of the activities. The particular positions and also the actions of users within the virtual world might be disclosed based on pre-configured user policies (e.g., simply to one’s listed buddies imported using their company websites like Facebook ). The issue of location privacy is much more important in these kinds of systems where private information is subjected to a potentially large group of people.
Additional privacy issues are based on the collection of environmental data by way of people carrying devices and, most importantly, data collected about the subject. In the authors discuss important security and privacy challenges associated with people-centric sensing. System Prototype Implementation Within this section, we discuss the general architecture from the system for that integration of activity information into virtual worlds for example Second Life and Map Services for example Google Earth.
We first describe ongoing work associated with the CenceMe platform, which acts because the core service for that retrieval and also the analysis of the sensed data within our architecture. Bridging CenceMe and Virtual Worlds. We provide a brief summary of the CenceMe platform to be able to understand the key problems that underpin the proposed architecture. For additional information on CenceMe make reference to . CenceMe is a general personal sensing system that allows members of internet sites to share their sensing presence extracted by way of mobile phones using their buddies. CenceMe has the capacity to provide details about the personal status of their users as well as their surroundings.
Similarly, by mining the sensed data CenceMe can extract patterns and options that come with importance inside a user’s life routines, like the current status when it comes to activity (e.g., as sitting, walking, standing, dancing, meeting friends) and physical or logical location (e.g., in the club, coffee shop, at the office or other significant places for that individuals).
We now have integrated the CenceMe back-end and also the Second Life servers using a connector component, produced for the Second Life Linux client, that receives inputs in the CenceMe back-end. The logical architecture from the system is shown within the Second Life Linux client Code from the LSL script triggering the mapping from the activities can be obtained as open-source software as an alpha version release. CenceMe may be used to interconnect virtual worlds and social media systems for example Facebook that are presently extremely popular.
Information could be fetched from social networks and displayed in virtual environments (e.g., presence details about the buddies/groups could be automatically reflected in Second Life). Virtual social events can as well be organized between people in the same social media community. Rather than using basic im, members could see what their colleagues and friends are doing (playing sports, walking, etc.) as well as their current geographical position using a three-dimensional visualization supplied by a virtual environment rendering engine such as the one utilized by Second Life.
This might replace the present simple but instead cryptic idle/away/online status icons and texts. There is currently no freely available API to create and move characters in map services using Google Earth or Microsoft Live Search Maps. A proposal for merging Google Earth and 2nd Life exists as far as inanimate objects are worried. We intend to integrate CenceMe which tools the moment an API is created available.
Design and Preliminary Evaluation from the Prototype. We design an initial prototype of the component bridging Second Life and CenceMe to higher understand the limitations in our systems design. The integration is dependent on the exploitation of virtual CenceMe objects the Second Life avatar can wear or carry. This object alters the avatar animation to mirror the actual activity state of the human user carrying a mobile phone that is running the CenceMe sensing application. The CenceMe object is implemented like a standard Second Life object that people program to speak with the CenceMe infrastructure via XML encoded requests and responses exchanged via HTTP.
Second Life CenceMe object behaviour is designed in LSL (Linden Script Language Figure 3. Second Life display of near-real-time inferred activities injected from CenceMe. Three states set through the CenceMe infrastructure are translated based on a user-defined mapping: sitting ! yoga-floating (left), standing ! standing (centre), and running ! flying (right). ). The interactions using the CenceMe infrastructure make use of the same CenceMe external application extension APIs as other application plug-ins (i.e., Facebook, Pidgin) which have been constructed (as described in ). Objects get a new state by polling the CenceMe infrastructure for a price that can be set by users (within ranges based on CenceMe administrators).
The avatar can disconnect from CenceMe if you take off the object. The interface between your CenceMe system and 2nd Life largely uses off-the-shelf technologies. Java servlets hosted inside a Tomcat server provide a set of APIs provided via XML-RPC for retrieving details about user activities from the MySQL database. The communication paradigm between your Second Life and CenceMe platforms that people adopt is pull-based, i.e., periodic requests are from a script running within the Second Life platform towards the CenceMe server. Our implementation uses this API to both retrieve inferred activities about CenceMe users in addition to a specification that defines how these inferences should be represented.
A library of potential Second Lifestyle is maintained (e.g., walking, teleporting, yoga-floating) that the user might want to map specific real-world activities. These relationships are imported and put on all incoming data. Real actions could be mapped to various actions within the virtual world (e.g., running mapped to flying). Each virtual object within Second Life retrieves data independently in the CenceMe server.
Each object is implemented while using LSL language. LSL may be used to specify object behaviour inside the Second Life environment. The code is published by Second Life servers into byte code having a maximum memory usage limit of 16KB. The byte code is executed by virtual machines running on these servers. LSL is definitely an event oriented language which is suited to implement state machine style solutions. Within our case, it’s used to implement reactions to alterations in real world activities by triggering modifications from the activities displayed within the virtual world.
Delay Measurements when the object is connected to the avatar, a periodic timer is initiated. Within our experiment, the time of the timer was 0.5 seconds. Your body of the timer causes the HTTP request to become made (begin to see the timer() body in Figure 2). An XML-RPC based request both the latest inferred action and also the user defined mapping from CenceMe inferred activities to Second Life actions is distributed. An event handler for that returning response (begin to see the http_response() block) first calls parseResponse() which processes the incoming response, returning a summary of potentially invoked avatar actions. Then, this output is supplied to a second function, induceAction() that takes these responses and determines which avatar action ought to be displayed. As described above, we make use of a clothing metaphor to supply an intuitive method for people to configure and control the mapping of the actions. In Figure 3, we show numerous screenshots of an avatar that has a CenceMe virtual object (in this instance, a pager – see inset).
Within the figures, we show three states set through the CenceMe infrastructure and translated based on a user-defined mapping: sitting ! yoga-floating (left), standing ! standing (centre), and running ! flying (right). Information about the activity classifiers are available in . Alternative CenceMe clothes and accessories include significant places t-shirts that display the emblem graphic representing an area of significance within the user’s life (i.e., the house, gym, pizzeria, etc.), and virtual mood rings that display the inferred emotional state of the baby. We also present some preliminary performance outcomes of the integrated system.
We show the typical delay experienced between performing the game in the real world and displaying the inferred activities in Second Life. The experiment includes a sequence of activities done by users in repeated sessions at different times during the day (at noon, 4pm, and 8pm ET).We average the information over 15 sessions. The succession of activities is standing still ! walking ! running ! standing still. The transitions between states are reported as step curves. We observe a typical delay around 30 seconds in reporting the game changes.
This delay is a result of the activity recognition process around the phones, the transmission from the inferred activity towards the back-end, its retrieval from CenceMe database, and it is display in Second Life. The delay is mainly caused by the periodic inference process running around the phones and also the uploading of the information towards the CenceMe back-end. The delay because of the transmission in the CenceMe back-end to the Second Life servers, and also the display from the activity information in Second Everyday life is negligible. We’re augmenting our current implementation to: investigate controlling movement within the geographical space; define appropriate privacy policy pages; and develop mechanisms to deal with periods once the avatar becomes disconnected in the physical user, or when inferred user state accuracy falls below a suitable bound.
Conclusion
Within this paper, we now have proposed to bridge real and virtual worlds using commercial cell phones equipped with standard sensors. We now have discussed numerous key problems that need to be tackled to understand our vision. We presented an initial implementation of the architecture, demonstrating an active system integration of CenceMe and 2nd Life. Towards the best of our knowledge, this is actually the first time that sensors utilized on everyday cell phones reflecting a person’s activity within the physical world continues to be expressed in Second Life.
Basic MySQL/PHP Interactivity – Exactly why is PHP/MySQL useful?
PHP is really a server-side scripting language that operates on a huge variety of platforms while offering web developers a strong tool to render dynamic HTML content. MySQL is really a popular, free relational database application that facilitates massive, rapid data storage and retrieval.
PHP may be used with many relational database applications, but provides a wide array of functionality aimed at interaction with MySQL. Overall PHP is really a forgiving programming language, it features a wide variety of built-in functions, includes a relatively simple syntax, and may be embedded straight into HTML. Both projects have extensive user bases, in addition to copious levels of documentation.
Practical Uses
PHP is a component of an estimated 20 million websites and A million web servers. Popular web applications for example Facebook, and Wikipedia utilize it to render their content. As pointed out above PHP is especially good at outputting dynamic HTML. So far as an expressive tool, PHP is excellent at formatting and display large types of media, in addition to performing logic operations on data. This is often information from the user or from another databases. PHP has additionally been extended by other, third-parties, types of this range from the PEAR application/object repository and frameworks for example CakePHP.
Basic: Running web server with PHP installed Basic knowledge of the PHP options enabled around the server Good text editor, designed with text highlighting functionality PHP manual MySQL manual Errors at verbose throughout the development process
The “Application”
The applying we will be building is definitely an image management system. The important thing functions the application must have are as follows:
- Offer an end user by having an HTML form that uploads images to some public image directory, and commits these phones a database together with addition descriptive meta-data.
- Give a method of recalling and editing this picture data.
- A listing view which allows the user to see the items in the images database. This application is going to be designed to highlight a few of the core functionality of PHP and MySQL. Though not terribly functional, it’s a good departure point for creating a more complex/usable application
Core Concepts
Fundamental Variable Types Understanding how to work with different types of variables and knowing their strengths, limitations and also the techniques accustomed to extract the values included in the variables. Of particular curiosity about this being active is the array and associative array (hash/dictionary etc) types accustomed to store multiple values by key.
Basic Control Structures Types of these include if/else statements, for performing different actions according to variable values, and looping mechanisms for example for each and while for iterating over complex variables for example arrays. Interpolation The ways that variables are interpolated or interpreted in strings using single or double quotes. Be aware of difference to ensure that when writing statements using strings you make sure that the PHP interpreter isn’t doing work.
Error management: Techniques accustomed to control the errors that occur once the connection functions are passed bad parameters, along with the techniques to assure the script ends gracefully. Particularly observe the techniques accustomed to both capture errors and interpret errors in addition to how to suppress them if needed.
Additionally it is vital that you decide upon the amount of verbosity your application must have, by wondering: “How much information concerning the process will i want to hand back to the user as feedback?” Code Reuse: Examine your code, whenever possible move repeatable actions into functions. This not just makes the code more legible to people who edit it once you, it helps you save from writing exactly the same logic over and over. Your time is valuable.
Basic Connection (basic_db_connection.php)
The initial step in using MySQL with PHP is establishing an association to the database server and upon success establishing an association to the specific database. The initial step is done with the mysql_connect function that will return a kind of link or resource towards the connection upon success, and can return false on failure.
In taking a look at this example make sure note the required arguments for mysql_connect and exactly how that errors are handled along with the way that success is registered. Following a successful link with the database server, the particular database must be selected using the mysql_select_db function, passing it the resource returned in the earlier function as well because the name from the database that you simply wish to use.
MySQL Connection as Function (basic_db_function.php)
Programmers are by habit (plus some might say necessity) a lazy bunch. The you are able to write quality code and obtain more things made by being lazy. This statement certainly seems paradoxical, but it is true. This situation shows a really basic the first step can take to ratchet in the lazy. By enclosing both mysql_connect and mysql_db_select calls inside a function, connections might be made over and over without the demand for rewriting exactly the same code again and again.
This methodology does apply to a multitude of problems and also the end result is (usually) more legible code, more maintainable code, less wasted effort and much more effective utilization of time.
Fetching (basic_db_select.php)
This file illustrates the strategy of querying the database for the records inside a particular table. While using mysql_query function and passing it a string variable which has the text from the SQL we can retrieve a question resource. This resource id may then in turn be employed to retrieve the particular results from the database. As before note the methods that errors are handled and pay particular focus on the way that the outcomes are retrieved by using a while loop.
Inserting (im_form_submit_example.php)
The insertion example may be the function “administration” element of our application. Utilizing PHP with the HTML form, we could both produce a new record and populate it with data in the form. Of note is that they specific way the shape is designed with the specific encoding kind of multipart/form-data and specially the way that PHP handles the uploaded files storing a number of information about it within the $_FILES array. Also introduced may be the specific syntax for inserting records in to the database.
Updating (image_manager/im_form.php)
This file represents the ultimate form for that actual image manager application. It serves a dual use, functioning as both uploader for creating new entries towards the database plus an editor, allowing a person to edit existing files within the database. Note how a file presents different content and performs different logic in line with the existence and items in the $_POST or $_GET variables.
Since the file does double duty, the functionality for uploading the files in addition to retrieving a particular file by unique ID. The file also utilizes the require_once function, including another file which has the details from the myDBConnect function as well as automatically produces the $dbConn resource to be used within the file.
Take not of how that the fields from the form are populated when the record has been retrieved in the database by ID. Be familiar with best practices regarding security and also the vetting of all data received externally of the application. An entire security audit of the application is outside the scope of the class but spot the use of the safeEscapeString() function utilized on all post and obtain data, to make sure that data is properly applied for the database in addition to eliminating the chance of a user running arbitrary SQL queries around the database.
Additionally, inside a production setting it might be wise to take away the MySQL login and password data in the DB function file, or move the whole file to some non-public directory. Then the file might be included again with the require_once() function after editing the include search path using the set_include_path() function.
Extend The applying
Here are some suggestions for extending the applying: Add a connect to the full size image within the database list view. Add keyword search and sort functionality. Add other sorting functions for name, date added, size. Add logic to ensure that all form fields are populated, and display a mistake if not. Extend the thumbnail generator to consider multiple types (gif,png,bmp) of images. Create functionality to delete records.