Massive Open Online Collaboration Testing
Based on the MOOCTEST platform, a multilingual online integrated development environment provides an online development and test learning environment for all users who have access to the system platform. Users can create the workspace of the project through WebIDE for online development, operation, submission and other operations. Users can use it anytime, anywhere, conveniently and quickly when they need, which greatly reduces the learning cost.
FEAT allow the users to access their automated testing techniques by implement some interface. So far, we access three automated techniques which have good compatibility: Monkey, AppCrawler and Appium.You can use FEAT to instrument and compile the Android source code, then use the techniques have been accessed to test them to get a test result, for evaluating the ability of these techniques.
BugHunter is a platform to solve the problem of low-quality and duplicate bug reports of crowdsourced testing. With the help of BugHunter, the crowdsourced workers can efficiently capture screenshot, write short description and create bug report. A series of bug reports are aggregated online and then recommended to the other workers in real time. The crowdsourced workers can (1) help review, verify and enrich each others bug reports; (2) escape duplicate bug reports; (3) be guided to conduct more professional testing with the help of collective intelligence. BugHunter can improve the quality of the final report and reduce the test costs.
CTRAS is a tool for automatically aggregating and summarizing duplicate crowdsourced test reports on the fly. CTRAS is capable of automatically detecting duplicates based on both textual information and the screenshots, and further aggregates and summarizes the duplicate test reports. CTRAS provides the end users a comprehensive and comprehensible understanding of all duplicates by identifying the main topics across the group of aggregated test reports and highlighting supplementary topics that are mentioned in subgroups of test reports. Also, it provides the classic tool of issue tracking systems, such as the project-report dashboard and keyword searching, and automates their classic functionalities, such as bug triaging and best fixer recommendation, to assist end users in managing and diagnosing test reports.
FuRong builds a bug model with complete context information, such as screen-shoots, execution events and logs from multi-devices, which are significant for developers, and then inducts a classification rule for bugs, which is the foundation for bug classification and deduplication. FuRong classifies bugs and removes some redundant bug information. FuRong also recommends a possible fixing solution for each type of bug. An empirical study of 8 open source Android applications with automated testing on 20 devices has been conducted. The preliminary results show the effectiveness of FuRong with the average accuracy of 93%.
Crowdsourced testing has become a new trend in testing Android applications. However, the crowds work separately without collaborations and guidance in existing platforms. In this paper, we leverage dynamic automated techniques and static analysis to promote crowdsourced testing. MoYe firstly constructs the window transition graph (WTG) with dynamic automated testing techniques, which provides the annotation of suspicious bugs. Then MoYe enhances WTG by extracting window transitions from APK files with static analysis techniques. Then, MoYe builds a recommendation engine based on the enhanced WTG model and user operations intercepted in real time to allocate different test tasks for different crowd workers. Further, MoYe provides real-time guidance for crowd workers to complete test tasks to verify suspicious bugs or explore new bugs. The preliminary experiments show that MoYe can improve the efficiency of crowdsourced testing. The demo video can be found at Youtube.
MAF, a plagiarism detection technology for test code, which relies on a constant similarity threshold to determine whether there is plagiarism between two pieces of test code. However, finding an appropriate threshold is never easy. We realize that a constant threshold cannot be used in every circumstance. To address this issue and make MAF more usable, we developed MAF-2 by applying a stable and reliable classifier based on Support Vector Machine classification algorithm. Experiments were conducted on three test code data sets, and the results show that MAF-2 can achieve plagiarism detection effectively. The video presentation of MAF-2 is available at Youtube and the source code can be downloaded at Github.
The fullname called "The Design and Implementation of Crowdsourced Feedback Review and Delivery System Based on Test Report Summary". Jie Mei has designed and implemented a crowdsourced review and delivery system based on the test report summary. The system has introduced a novel mechanism, which is based on reporting summarization to solve the problems of the crowdsourced feedback review.
The online crash analysis system fully adapts to the mainstream categories of Android applications in the market. Extensive experiments have been performed for 10 different open-source applications 20 Android devices. Experimental results demonstrate that, the system implemented in this thesis not only can automatically capture Android application crashes, but also can effectively classify, deduplicate and visualize crash information. The classification accuracy rate achieves 88.1%, and deduplication rate achieves 60.7%.
GUI TSRS designs and implements an Android application GUI test script repair system, which improves reusability by repairing GUI test scripts that have failed due to version updates. The system divides the repair process into four parts. Firstly, the GUI components of Android applications are extracted by using the automated traversal tool and the event flow graph model is established.
QCS improves the quality of bug reports and the quality awareness of crowd workers, enables managers to identify the quality of bug reports and malicious crowd workers quickly, and makes it easy to review massive bug reports for managers. Also, it promotes quality control study in collaborative crowdsourcing mode.
AGS designs and implements an automatic generation system for bug report of mobile applications to solve the problems of low readability of logs and low efficiency of bug location in automated testing. By mining multiple logs generated by automated testing, this system builds a bug model containing complete context information, including operation sequence, stack information, bug screenshots, etc.
The system introduces the Isolation Forest algorithm to implement data annotation preprocessing, thereby reducing the workload of data labeling for operation and maintenance personnel. Furthermore, the accuracy of anomaly detection and trend prediction is improved by iteratively updating data labels and resetting models. The system is mainly divided into four modules. The monitoring module is responsible for the collection and storage of monitoring data.
BLS selects a total 1243 bug reports created before October 16,2018 about the open source project Zookeeper to verify the bug location accuracy of this system initially. The hit rate of Top1 in the bug localization result is 52.55%, the hit rate of Top 5 is 78.94%, and the hit rate of Top 10 is 87.45%. In addition, we simulated 500 users concurrently accessing 7 common interfaces in the system performance test. The results show that the average response time of the seven interfaces is 247ms, the highest is 447ms. All responses take less than 500ms and the error rate is 0.
CTSR&RS selects 10 mainstream mobile devices and 5 mobile applications to analyze and verify the system availability. The experiment proves that the record process of this system has better support for different devices under the Android platform, and the script cross-device replay success rate is over 80%. The cross-platform replay mode has a success rate of about 60%. The system has been deployed for internal mobile application testing. Through a single record of test scripts and cross-platform replay of scripts, it effectively reduces the number of test script maintenance and the cost of automated testing.
The platform solves the problem of insufficient human and device resources through online real machine control and bug crowdsourced verification. Crowd workers can remotely record and replay test scripts in the device, and they can modify and perfect test scripts. Bug crowdsourced verification allows task requesters to verify bugs through crowdsource and improve verification efficiency based on the results of verification submitted by crowd workers and the statistical distribution of results.
CQCS mainly expounds the design and implementation of the code quality control system for development team based on Git. The main functions of the system are responsible by three modules together, including the code quality control module that extracts the quality characteristics of the code, the Git control module that manage and excavates the evolution history of the project, and the team’s code quality evaluation module that integrates the data to give the evaluation results. In order to ensure the well scalability of the system, the module transmits information through metadata, and invokes the initiator module to fit the metadata from the invoked module through the data converter.
This system is realized by mainstream frameworks, with Angular2 as the frontend framework, Spring Boot as the back-end framework, Redis as the query cache, and MongoDB as the database. Similar report recommendation is implemented by Word2Vec and WMD algorithm. The audit task recommendation is implemented using a model-based collaborative filtering approach. The test page recommendation is implemented using the multi-source shortest path method based on users’histories.
Intelligent Code Recommendation System Based on Structure Embedding Analysis
This tool relies on the Mooctest WebIDE system to update and optimize, and implements an
intelligent code recommendation system based on structure embedding analysis. When users
perform programming exercises or exams, they access the system through a browser and use
an intelligent editor for online programming. The tool will monitor the user's
programming behavior and grab code information for analysis and recommend code fragments
to users based on the analysis results. Code recommendation relies on a large amount of
source code data, so it is necessary to build a corpus based on a large amount of source
code and preprocessing. The candidate code fragments with high similarity are trained by
word embedding. Sentence embedding is performed to obtain a vector representation. At
present, the tool has replaced the original Mooctest WebIDE system for online use.
Experiment shows that the system improves the programming experience of 82.72% users on
average.
Test Recommendation System Based on Slicing Coverage Filtering
This tool relies on Mooctest WebIDE to design and implement a Test Code Snippets
Recommendation System based on Slicing Coverage Filtering. This tool uses Wala as a
program slicing tool. AST program analysis technology is used to merge the code snippets
with the project template. The OpenClover tool is used to analyze the test coverage of
the code snippets and store it in the corpus. During the user's test learning, the
system will analyze the user's test coverage information in real time, and use the test
coverage vector to calculate the Jaccard vector similarity filter to obtain the relevant
code snippets in the recommended corpus. The tool has constructed a recommended corpus
containing 11 original questions and more than 2,200 test code snippets.
WebTester
This tool relies on Mooctest WebIDE to design and implement an online test development
system based on homologous code matching. The LSP protocol is used to add multi-language
intelligent code prompts. At the same time, a test automation generation scheme is
proposed. WebTester collects historical data of open source websites and examination
platforms to build a corpus of test codes. The structure and text information of the
code under test is extracted; WebTester combines string matching, spelling correction,
near-sense search, program similarity analysis and other means to measure homology. The
test cases of the homologous method in the corpus are searched and modified, and concise
and usable test code for the method to be tested is generated. For projects not covered,
the system integrates the optimized Evosuite tool to provide users with basic test cases
for the target method. Statistics shows that WebTester reduces the average processing
time of a single project from 65s to 20s, and the throughput is greatly improved.
Middle-end System for Android Application Automation Test
Combined with the high efficiency of automated testing and the idea of universality and
flexibility of testing Middle-end system, we proposes and implements an Appium-based
Middle-end system for Android application automation test. This tool first analyzes the
APK file that need to be tested, and then selects the currently idle Android deivces and
performs an automated traversal test through the Appium framework and ADB tools. At the
same time, the tool supports dynamic execution of custom Java scripts through Groovy to
increase system scalability. By specifying different report generation services, the
tool can analyze the intermediate data from the perspective of performance,
functionality, stability, robustness, and compatibility to generate multi-dimensional
test reports. Also, the tool supports custom test report analysis tools and the tools
can be hot plugged. 0At present, the system has landed on the Mooctest cloud platform,
providing APK grading services for the education version, and stable and reliable
automated testing services for the enterprise version.
BREGAT
BREGAT is used to generate structured bugs for multi-device automated testing results,
classify and deduplicate bugs via the defined taxonomy. We first defined the GUI
inconsistency and device inconsistency based on multi-device automated testing results,
and proposed a structured bug model that can be used for inconsistency analysis. Through
the manual review of the real automated testing results of 50 applications on 20
devices, analyzing the relationship between bugs and inconsistencies, confirming the
root cause of bugs, and extracting common log patterns of similar bugs, we constructed
an extensible bug taxonomy with inconsistency labels, which contains a total of 67 bug
categories. Finally, heterogeneous data in the testing results such as screenshots, test
operations, and device logs are combined to generate reproducible and easy-to-understand
bug reports. BREGAT removes 97% of duplicate bugs, and the deduplication precision
reaches 100%, which is more efficient than existing tools.
ColinTest
ColinTest is an iterative Android application automated testing system, which
automatically records, extracts, and fuses the user operation flows of the test users,
and enters them into the testing tool. Through iterations in testing process, user
operation information is introduced into the tool, and the output of the tool is passed
back to the user to generate the next round of user operation information. By modifying
the Appium framework, ColinTest automatically obtains the user operation flows in the
background. The experimental results show that the test results of the system have been
significantly improved compared to the case without the introduction of user
information. When the test time is set to one hour, the average code coverage rate
reaches 37.83%, which exceeds Monkey under the same conditions, which average code
coverage is 28.90%. In addition, the test results after the introduction of the
information completely included the coverage of the user or the tool when it was tested
alone, and no coverage omissions were generated, which undoubtedly proved the usability
of the system.
Crowdsourced Testing Requirement Generation System Driven by Android Automatic Testing
This tool implements a crowdsourced test requirement generation system driven by Android
automatic testing. The tool processes the automatic test data of the target application
and extracts the exceptions triggered during the test. Based on test operation sequences
and screen shots, workers are guided to find exceptions in different environments by
reproduction steps. Using decompilation and static analysis techniques, the types of
components involved in the conditional branch statements in the source code and the
windows not covered in the automatic test are extracted to guide crowdsourced workers to
explore new exceptions. 10 Android open source applications were selected and controlled
experiments were carried out on 15 popular mobile devices. The experimental results
showed that by guiding the crowdsourced workers with our generated requirements, 30%
more exceptions could be triggered and 15% more coverage rate was achieved than the
control group. In summary, this tool provides a service for generating crowdsourced
requirements and accepting crowdsourced test results.
Report Generation Service in Web Application Automation Test System
According to the characteristics of Web application testing, test report service
implements the Web application automation test process based on the monitoring of test
execution, including the generation of test results and the display of test reports,
software quality assessment reports. For the characteristics of widely use and high
complexity of Web applications, we selects 50 representative websites from nine
different industries to experiment on and evaluate test report service. The experiment
shows that the service can provide distinct and easy-to-use test reports. The success
rate of the test script generated by the test report service reaches 99.8%, the
recurrence rate of existing vulnerabilities reached 89.4%, and the accuracy rate of
vulnerabilities classification reached 97.5%. The experiment also finds that using
scripts to reproduce vulnerabilities is nearly three times more efficient than manually
reproducing vulnerabilities. Currently, the service has been integrated on the Mooctest
platform and is running well, providing testers with good Web application automation
test report services.
Execution Service in Web Application Automation Testing System
This tool is used to solve the needs of automated testing of complex dynamic Web pages.
Testing vulnerabilities reports and software quality assessment can be generated. The
service applies the idea of Web page state detection based on finite state machine to
test execution, searches the web page state space, and generates executable test paths.
The service is based on microservice technology and distributed technology. We selected
common types of websites and conducted preliminary verification of this automated
testing service. Each application can detect an average of 89 web application states,
and on average can find out 83 vulnerabilities on each application. The execution
success rate of the test scripts generated by the state diagram constructed by the test
execution service is 99.8%. The vulnerabilities recurrence rate after script execution
is 89.4%, and the vulnerabilities classification accuracy rate is 97.5%. At the same
time, the testing service can improve the testing efficiency by 2.12 times compared to
manual.
Analysis Service in Web Application Automation Testing System
This tool is mainly for the defect scanning and defect analysis of web application
testing, and provides intelligent solutions. The system relies on multi-source“black
box”data of the system under testing, such as related logs, screenshots and so on, which
are recorded by the execution service and scheduling component responsible for automated
traversal testing. With methods such as clustering algorithms and feature detection, a
web application related defect system is constructed, realizing and integrating some
system defect detectors corresponding to relevant data. Finally, the system discovers
defects through detection, and constructs test report of the application to be tested.
Experiment based on 50 real online websites shows that the system can effectively
discover various web application defects such as broken links, resource loading
failures, server-side errors, JavaScript errors and so on. At the same time, both the
accuracy rate of system defect classification and the correct rate of export test script
execution exceeds 95% based on manual sample and review.
Multidimensional Evaluation System for Software Testing（META）
This system aims to solve the problem of how to systematically evaluate the
effectiveness of students' test. According to the talent training objectives of colleges
and universities and the requirements of enterprises, we constructs a multi-dimensional
evaluation system to evaluate students' testing effect from various aspects on the basis
of 7 evaluation indicators in combination with test products and test behaviors, aiming
at three test types of developer unit test, Web application automation test and mobile
application automation test.
The tool we roposed aims to more comprehensively evaluate the test effects of
students. Its evaluation indicators has been widely used in the software testing
contest hosted by the teaching platform, and the evaluation result has been unanimously
recognized by the expert group of the contest. The system has been used in the
software test teaching platform, through a variety of evaluation of students' test
effect, in order to help students find shortcomings, so as to promote their
progress.
MAF
We deeply analyzes the test code designed under the unit test framework and production
code, explores the potential differences between them, designs and implements a test
code plagiarism detection system based on program slicing. In order to achieve
high-quality test code plagiarism detection, this system innovatively proposes a static
two-way program slicing technology, extracting effective test fragments from
non-standard test code based on the method under test, further calculates the similarity
between test fragments, and carrys out plagiarism analysis based on similarity. After a
series of strict system tests, the system has achieved functional and non-functional
requirements, which measures up to the anticipative assumptions. Through the analysis
and verification on a large number of real datasets, the experiment shows that the
system can effectively detect the plagiarism of test code, and the system has good
robustness through performance analysis.
Expressive Ability Evaluation System Based on Distributed Task Queue
This tool is an expressive ability evaluation system based on distributed task queue. It
can provide evaluation services for ordinary users and provide related system management
functions for administrator users. The answers should be made by speech in Mandarin. The
system will record, store, and score users' speeches. The system is divided into four
modules: evaluation module, permission management module, questions management module,
and asynchronous tasks module. The expressive ability evaluation system meets the
expected performance requirements, and provides a good support for the quick expressive
ability test method. The system can reduce the capital, site, and labor costs in the
expressive ability test, improve the test efficiency and test result's objectivity. It
also brings a more convenient test experience to users. It can be used to provide
reference for recruitment and to provide a evaluation or exercise platform for people
who want to improve their expressive abilities.
Problem Construction and Analysis System for Expression Ability Evaluation
This tool focuses on the entire process of the core element of the "problem kit" in the
evaluation, from generation and initialization to use and analysis and optimization.
First, the system needs to regularly crawl network resources and initialize it as a
backup question. Second, the system will use audio analysis and text analysis methods to
analyze the user's expression level and provide brief comments. Third, the system needs
to use the answer data submitted by the user and use statistical learning method to
optimize the parameters of the problem kit and improve the stability of the model. In
addition, the system will also provide the necessary support so that professionals can
intervene in the problem life cycle, generate new problems from the alternate corpus, or
modify the problem parameters artificially. The experimental results show that the
number of samples within 5 points between the system's scoring results and the expert's
score exceeds 75%, indicating that the system can quantify the evaluator's expressive
ability to some extent.
Human-machine Collaboration-Based System for Java Vulnerability Scanner
In order to reduce the false positive ratio of current vulnerability scanners and save
maintenance costs for developers, we design a human-machine collaborative Java bytecode
vulnerability scanning system. It analyzes static vulnerability scanners and common
false positive vulnerabilities. This tool implements bytecode context extraction, code
feature extraction, and related machine learning classification models. It also
integrates crowdsourcing auditing and combines the vulnerability scanning requirements
in actual scenarios to implement the system. The human-machine collaborative Java
bytecode vulnerability scanning system can provide better vulnerability scanning
service. Experiments on the OWASP dataset show that the system can reduce the false
positive ratio by 22%, when the recall is 95.39%. The result of experiments, which are
based on randomly selected open source projects on the GitHub, shows that the system can
effectively reduce the false positive ratio of traditional static vulnerability scanners
while ensuring a low false negative ratio.
C++ Source Code Vulnerability Static Scanning System
C++ source code vulnerability static scanning is digging the potential vulnerability of
C++ source code using taint analysis and data flow analysis without running the program.
In order to improve the developer vulnerability review process and reduce the difficulty
of vulnerability review, the C++ source code vulnerability static scanning system
urgently needs to reduce the false positive ratio of false negatives to assist
developers in delivering more robust code. This system innovatively introduces an
iterative false alarm filtering mechanism based on machine learning to reduce false
positive ratio in vulnerability scanning. The F1 value of the system is increased by 30%
and 22% compared with TscanCode and Cppcheck, and it can effectively reduces false
positive in the C++ source code vulnerability static scanning. This system improves the
availability of static scanners for C++ source code vulnerabilities, reduces the number
of false positive vulnerabilities, reduces the burden on developers to review
vulnerabilities, and provides guarantees for delivering highly reliable code.
The Design and Implementation of Static Code Analysis System based on Machine Learning for Java
This system aims to apply academic research to the real world. For one of the most
common languages in Web development, Java, this system utilizes taint analysis, program
slicing, and BLSTM to provide more accurate code analysis services for development or
security engineers. In terms of taint analysis, the system uses a large number of rules
in Find Security Bugs to ensure low false negatives. In addition, it can give taint
propagation paths to make the report more readable. In order to ensure slicing
efficiency and stability, this system optimizes the slicer for the actual Jar package
and proposes an idea called segmented slicing. The experimental results show that the
system can obtain more accurate scanning results within an acceptable scanning time. In
terms of efficiency, the system optimizes traditional slicing to ensure that the scan
time of each project does not exceed 1 hour. For accuracy, the system's precision rate
reaches 90.53%. In other words, compared to Find Security Bugs, the system has
eliminated 25.44% false positives, which greatly reduced the code audit work.
Crowdsourced Feedback Review Tasks Distribution System Based on User Features
Based on the crowdsourced review platform, a crowdsourced review task allocation system
based on user characteristics is proposed. This system fully considers the user's
reputation characteristics, ability characteristics and behavior characteristics, and
designs two modes of static task allocation and dynamic task allocation. For static
review tasks, the static task assignment mode is invoked, and each user is assigned the
same number of test reports to be reviewed according to the needs of crowdsourcing
review tasks. For dynamic review tasks, the system will call the dynamic task assignment
mode, use the collaborative filtering algorithm to generate a user task recommendation
list, and generate a task heat list based on task assignment and complete details. After
the comprehensive calculation, the user will be assigned a test report. The average
review coverage rate reaches 100\%, the accuracy rate of the static task review result
reaches 87.8\%, and the accuracy rate of the dynamic task review result reaches 95.12\%.
Therefore, the system has a high efficiency.
Unified Task Allocation System for Crowdsourced Testing
This system focuses on selecting the desired tasks for crowdsourced workers and
completing the design and implementation of a unified task allocation system for
crowdsourced testing. Relying on a domestic well-known collaborative testing platform M,
this system creatively proposes a multi-objective dynamic task allocation mechanism
based on priority to improve the testing efficiency. Specifically, the system analyzes
the fitness between tasks and workers from four aspects: test ability, experience,
willingness and credibility, and uses uses breadth-first test traversal and timer to
achieve reasonable task allocation.At the same time, the task pricing mechanism based on
user choice will greatly stimulate the enthusiasm of crowdsourced workers to complete
the their tasks with high quality on the premise of fully ensuring workers' free choice.
Therefore, according to these requirements, the system can be divided into four
functional modules: test task adaptation, dynamic task allocation, test task pricing and
records of the implementation status of crowdsourced workers.
Crowdtest Data Provenance System Based on Consortium Blockchain
In order to achieve reliable traceability of public test data, a Consortium blockchain
composed of crowdtesting demanders, crowd workers, and crowdtest platform parties is
established, and a crowdtest data provenance system based on consortium blockchain is
designed and implemented. This system collects crowdtest data in real time and attaches
source information as provenance data for consortium blockchain storage. The stored
process is verified by each participant as a transaction in the consortium blockchain
network and is automatically completed by the smart contract. Traceability data is
maintained as a transaction ledger by multiple parties and is difficult to tamper with.
The test results show that the system can provide reliable source data for crowdsourcing
testing based on consortium blockchain. The consortium blockchain in this system can
still maintain good availability under the transaction throughput of 50tps. This system
provides crowdtesting data traceability services for crowdsourcing testing, improves
trust among participants, and contributes to the development of crowdsouring testing.
Crowdsourcing Testing Assets Confirmation System Based on Blockchain
Relying on the crowdsourced testing platform of MoocTest, we designed and implemented a
crowdsourced testing assets confirmation system based on blockchain technology, aiming
at solving the problems existing in the current crowdsourced testing. By storing the
data involved in the crowdsourced testing process into the blockchain, and applying the
technologies of chain data storage, consensus algorithm, intelligent contract and so on,
the system realizes the mutual trust of multiple participants in the data level, and
makes the data in the flow of the crowdsourced testing process open and transparent.
The results show that the system can realize the safe storage, real-time tracing and
assets confirmation of the data. The functions of the system includes the data storing,
querying, verifying and asset confirming. In terms of system performance, the system
throughput can reach 300tps, which can support the business requirements in real
scenarios. In terms of system security, there are no security holes in the blockchain
smart contract of the system, and the system can guarantee the fault tolerance rate of
1/3.
PerTether
By analyzing Ethereum's PoW consensus protocol and Gas mechanism, we finally recognized
two performance impact factors for private Ethereum blockchain: Difficulty, Gas Limit.
We introduce fault injection technology. By analyzing the common faults of distributed
systems along with blockchain characteristics, we propose four types of faults on
private Ethereum blockchain: application, consensus, smart contracts, network.
Afterwards, we divide each failure into three levels according to the severity of the
fault. Accordingly, we implement a performance testing System for private Ethereum
blockchain via fault injection.
Through fault injection technology, real-world faults can be simulated in the test
environment. We design and perform two sets of experiments, including impact factor
verification experiments and fault injection verification experiments. In the impact
factor verification experiment, with the change of the factor, the throughput decreased
and the latency increased significantly. In the fault injection verification experiment,
the performance of Ethereum showed a downward trend during fault injection. The
experimental results show that the system can reflect the performance and stability of
Ethereum under fault injection through performance indicators.
MuSC
We design and implement a mutation testing system for Solidity language. Specifically,
we studies the language characteristics of solidity from the aspects of keywords, global
variables/functions, exception detection and contract vulnerabilities, and propose 16
special mutation operators for Solidity. MuSC can generate a large number of mutants
efficiently and accurately at the level of abstract syntax tree, and supports automatic
deployment and testing of contracts based on the Truffle framework. In addition, MuSC
also provides many other functions such as mutant display, custom test chain creation,
test report generation, etc.
Experimental results show that the method based on mutation testing is superior to the
coverage-based method in defect detection rate (96.01% vs. 55.68%), indicating that
mutation testing can better measure the adequacy of ESC testing than code coverage. In
addition, we browses and classifies 729 real ESC error reports, and finds that 117 of
them are related to the Solidity special mutation operator, indicating that the new
mutation operator proposed in this paper can effectively reveal true defects.
Quality Inspection Platform for Civil Judgement Documents
Based on the knowledge of data quality and information quality assessment, this product
divides the data into structured and unstructured according to the format. Based on the
actual needs of the scene, the design of assessment indicators is completed from
subjective and objective dimensions. For structured information, mainly including tag
data which can be extracted by keywords. In this paper, data quality is measured by the
commonly used measurement method, combined with the measurement dimension mentioned in
the objective information theory. It covers seven categories: delicacy, delay,
authenticity, integrity, consistency, readability and accuracy. 15 detailed evaluation
indexes are designed according to the requirements of civil judgment documents. For the
unstructured information of text type in data, we pay more attention to the quality of
semantic and pragmatic information contained in it. With the help of supervised machine
learning and deep learning technology, we take the large number of data accumulated in
the system as positive training samples with high information quality to simulate the
artificial information quality evaluation process.
Judicial Text Data Automatic Generation System
We design an automated system for generating judicial text data and implements it,
including training and testing data generation module. The training data generation
module is used to provide data augmentation services for the judicial deep learning
model, increasing the high-quality judicial text training data and improving the
prediction accuracy of the model. We design two generation methods based on rule and
Variational Auto-encoder. Combining with the characteristics of judicial text, we
propose an augmentation method for judicial text in rule-based generation. The
generation method based on Variational Auto-encoder applies the Variational Auto-encoder
in the field of text generation, learning the low-dimensional features of judicial text,
adding noise and reconstructing new text with similar distribution.
We conducted extensive experiments to prove that the two training data generation
methods provided by the system are effective, and can increase the accuracy of the crime
prediction TextCNN model from 81.91 % to 83.31 %.
Model Evaluation System Driven by Judicial Data Quality
After a comprehensive analysis of the background, current situation and system
requirements, we design and implement a model evaluation system driven by judicial data
quality, which is mainly divided into data interaction, document analysis, quality
detection and model evaluation modules. Through this system, users can select or upload
machine learning models, select the built-in documents or upload the data set of
judgment documents, carry out judicial intelligence classification tasks such as law
articles prediction, and calculate the model evaluation index. Among them, KNN, SVM and
Naive Bayes are the basic machine learning models built in the system, and the model
evaluation indexes used in the system are Accuracy, F1 score, KS value and PSI. For the
built-in or uploaded judgment documents, the system will automatically perform field
parsing, label classification and feature extraction, and generate quality inspection
reports. The quality attributes include interpretability, relevancy, accuracy and
consistency.
System for Testing Judicial Case Screening System
The main work of this product is to set up a series of multi-dimensional case screening
test criteria and to build a highly functional testing platform for case screening
system. This system conducts the test of case screening system in two aspects: the model
level and the system level. The relevant machine learning models for the case screening
system are evaluated by basic and extensive metrics based on the specific data set.
Multi-dimensional similarity metrics test the system-level interfaces of the case
screening system. The testing system eventually works well in the form of a web
application. The user can access the system through a browser to manage data set and
model files, conduct testing tasks automatically to generate corresponding testing
results. This testing system could be highly practical for developers and testers in the
legal field.
TOP