Abstract

Weighted Frequent Itemset (WFI) mining is animportant model in data mining. It aims to discover all itemsetswhose weighted sum in a transactional database is no less than theuser-specified threshold value. Most previous works focused onfinding WFIs in a transactional database and did not recognizethe spatiotemporal characteristics of an item within the data.This paper proposes a more flexible model of Spatial WeightedFrequent Itemset (SWFI) that may exist in a spatiotemporaldatabase. The recommended patterns may be found very useful inmany real-world applications. For instance, an SWFI generatedfrom an air pollution database indicates a geographical regionwhere people have been exposed to high levels of an air pollutant,sayPM2.5. The generated SWFIs do not satisfy the anti-monotonic property. Two new measures have been presentedto effectively reduce the search space and the computationalcost of finding the desired patterns. A pattern-growth algorithm,called Spatial Weighted Frequent Pattern-growth, has also beenpresented to find all SWFIs in a spatiotemporal database.Experimental results demonstrate that the proposed algorithmis efficient. We also describe a case study in which our modelhas been used to find useful information in air pollution database.

Abstract

Weighted Frequent Itemset (WFI) mining is an important model in data mining. The popular adoption and successful industrial application of this model has been hindered by the following two obstacles: (i) finding WFIs is a computationally expensiveness process as these itemsets do not satisfy the downward closure property and (ii) lack of parallel algorithms to find WFIs in very large databases (e.g. astronomical data and twitter data). This paper makes an effort to address these two obstacles. Two pattern-growth algorithms, Sequential Weighted Frequent Pattern-growth and Parallel Weighted Frequent Pattern-growth, have been introduced to discover WFIs efficiently. Both algorithms employ three novel pruning techniques to reduce the computational cost effectively. The first pruning technique prunes some of the uninteresting items by employing a criterion known as cutoff weight. The second pruning technique, called conditional pattern base elimination, eliminates the construction of conditional pattern bases if a suffix item is an uninteresting item. The third pruning technique, called pattern-growth termination, defines a new terminating condition for the pattern-growth technique. Experimental results demonstrate that the proposed algorithms are memory and runtime efficient, and highly scalable as well.

Abstract

Efficient discovery of interesting patterns using parallel algorithms is an actively studied topic in data mining. A key research issue related to this topic is data segmentation, which influences the overall computational requirements of an algorithm. This paper makes an effort to address this issue in correlated pattern mining. Two novel data segmentation techniques, ‘database segmentation’ and ‘transaction segmentation,’ have been introduced to discover the patterns efficiently. The former technique involves segmenting the database into multiple sub-databases such that each sub-database can be mined independently. The latter technique involves segmenting a transaction into multiple sub-transactions such that each sub-transaction can be processed as an individual transaction. The proposed techniques are algorithm independent, and therefore, can be incorporated into any parallel algorithm to find correlated patterns effectively. In this paper, we introduce map-reduce based pattern-growth algorithm by incorporating the above mentioned techniques. Experimental results demonstrate that the proposed algorithm is memory and runtime efficient and highly scalable as well.

Abstract

Preablative stimulated thyroglobulin (ps‑Tg) is an important investigation in the follow‑up of patients with Differentiated thyroid cancer(DTC) after surgery. Levels of ps‑Tg >2–10 ng/ml have been suggested to predict metastasis to cervical and extracervical sites. There is still debate on the need for routine iodine whole‑body scan (131I WBS) in the management of low‑to‑intermediate‑risk DTC patients. Objective: We analyzed our data of patients with DTC who underwent total thyroidectomy to discuss the predictability of ps‑Tg on metastatic disease on the 131I WBS. Materials and Methods: Retrospective analysis of patient records. Results: One hundred and seventeen patients with DTC (95 papillary thyroid cancer [71 had classic histology, 8 had tall cell variant, 16 had follicular variant] and 22 follicular thyroid cancer [18 minimally invasive, 2 hurtle cell, and 2 widely invasive cancers]) had undergone total thyroidectomy. All these patients underwent ps‑Tg assessment and an 131I WBS. About 65% of them went on to have radioiodine ablation along with a posttherapy 131I WBS. We divided the cohort into four groups based on their ps‑Tg levels: Group 1 (ps‑Tg <1), Group 2 (ps‑Tg 1–1.9), Group 3 (ps‑Tg 2–5), and Group 4 (ps‑Tg >5). None of the patients in Group 1, 7% of those combined in Groups 2 and 3 (2 out of 28 patients), and 26% (12 out of 47) of those in Group 4 had either cervical or extracervical metastasis. Those with extracervical metastatic disease to lungs and bones had a mean (standard deviation) ps‑Tg value of 436 (130) and median of 500 ng/ml and those with cervical metastatic disease had a mean Tg value of 31 (64) and median 6.6 ng/ml. Conclusions: A ps‑Tg value in the absence of anti‑Tgantibodies <1 ng/ml reliably excludes metastatic disease in DTC, while a value >5 ng/ml has a 26% risk of having either cervical or extracervical metastasis.

Abstract

Weather-based decision support systems (DSSs) are being built to improve the efficiency of the production systems in the domains of healthcare, agriculture, transport, governance and so on. Normally, a weather con-dition (WC) is represented by the statistical values of weather variables for a given duration (e.g. a day or a week). In a weather-based DSS, given a WC, the domain experts prepare the appropriate suggestions to improve the efficiency of the stakeholders. Normally, once the domain experts prepare a suggestion for a given WC that belongs to certain period (e.g. year or season), there is a scope to reuse the same suggestion for the similar WCs of other period(s). As a result, the performance of the DSS could be improved. In this paper, to improve reuse, we have proposed a notion of category-based WC (CWC) which is formed by using the categories of weather variables in the respective domain. By considering the context of agromet advisory service operated by India Meteorological Department (IMD) and the corresponding weather categories provided by IMD, we have ana-lyzed the extent of reuse among CWCs by conducting the experiments on 30 years of weather data collected at Rajendranagar, Hyderabad, Telangana state. The experiments are conducted by considering two types of CWCs with the duration equal to one day and five days. By varying the number of weather variables in CWC from one to five, we have computed the extent of reuse among CWCs of different periods of the following period types: year, season, and phenophases (i.e., growth stages) of the Rice crop. The results show that there is a significant similarity among the CWCs of the given period and the CWCs of the preceding periods of each period type. For any domain including agriculture, the results provide an opportunity to improve the efficiency of weather-based agricultural DSSs by improving the reuse of the weather-based suggestions

Abstract

Many domain specific decisions of our day-to-day lives are dependent on weather. Weather and its variables have a significant impact on crop gro wth stages and crop producti vity.Weather forecast service has been started by In dia Meteorological Department (IMD) from the year 1945. To help the farming community, IMD has started pro viding district level agro-meteorological advisory based on the district-level Medium Range Forecast (MRF) f rom the year2008 fo r all districts through 130 agro-meteorological field units (AMFUs). Based on the MRF for five days, each AMFU is providing crop-s pecific agromet advice for the districts covered by that AMFU. To impro ve the effectiveness, IMD has started block level MRF on an experimental basis from the year 2015. Based on block level MRF, IMD is planning to provide block-level agromet advisory based on the block-level MRF. In such a situation, agromet scientisst haveto prepare agromet advisory for several blocks and crops at the earliest after receiving the block level MRF which will be disseminated to the farmers and stakeholders. In this context, research efforts have to be made to develop efficient system to enable agromet scientists to prepare the block-level agromet advice based on the block-level MRF. Normally, it can be observed that the weather and cro ps of nearby blocks may not vary significantly. S o, there is an opportunity to reuse the agromet advice prepared for one block to the nearby blocks. In this paper, we have pro posed an IT- based framewo rk to enable the agromet scientists to prepare the block level agromet advisoryby exploitingthe fact that the agromet advice which has been prepared for one block can be reused for another block if the weather condition and the crop is the same. The proposed framework pro vides the scope to exploit the reuse of agromet advisory prepared for one block toother blocks and could improve the efficiency of agromet scientists for the preparation of block-level agromet advisory based on the MRF

Abstract

t. In addition to maximizing the revenue, retailers also aim at diversifying product offerings for facilitating sustainable revenue generation in the long run. Thus, it becomes a necessity for retailers to place appropriate itemsets in a limited k number of premium slots in retail stores for achieving the goals of revenue maximization and itemset diversification. In this regard, research efforts are being made to extract itemsets with high utility for maximizing the revenue, but they do not consider itemset diversification i.e., there could be duplicate (repetitive) items in the selected top-utility itemsets. Furthermore, given utility and support thresholds, the number of candidate itemsets of all sizes generated by existing utility mining approaches typically explodes. This leads to issues of memory and itemset retrieval times. In this paper, we present a framework and schemes for efficiently retrieving the top-utility itemsets of any given itemset size based on both revenue as well as the degree of diversification. Here, higher degree of diversification implies less duplicate items in the selected top-utility itemsets. The proposed schemes are based on efficiently determining and indexing the topλ high-utility and diversified itemsets. Experiments with a real dataset show the overall effectiveness and scalability of the proposed schemes in terms of execution time, revenue and degree of diversification w.r.t. a recent existing scheme.

Abstract

Utility mining has been emerging as an important area in data mining. While existing works on utility mining have primarily focused on the problem of finding high-utility itemsets from transactional databases, they implicitly assume that each item occupies only one slot. However, in many real-world scenarios, the number of slots consumed by different items typically varies. Hence, this pa-per considers that a given item may physically occupy any fixed (integer) number of slots. Thus, we address the problem of efficiently determining the top-utility itemsets when a given number of slots is specified as input. The key contributions of our work are three-fold. First, we present an efficient framework to determine the top-utility itemsets for different user-specified number of slots that need to be filled. Second, we propose a novel flexible and efficient index, designated as the STUI index, for facilitating quick retrieval of the top-utility itemsets for a given number of slots. Third, we conducted an extensive performance evaluation using real datasets to demonstrate the overall effectiveness of the proposed indexing scheme in terms of execution time and utility (net revenue) as compared to a recent existing scheme.

Abstract

Periodic Frequent patterns (PFPs) are an important class of regularities that exist in a transactional database. In the literature,pattern growth-based approaches to mine PFPs have be proposed by considering a single machine. In this paper, we propose a Map-Reduce framework to mine PFPs by considering multiple machines. We have proposed a parallel algorithm by including the step of distributing transactional identifiers among the machines. Further, the notion of partition summary has been proposed to reduce the amount of data shuffled among the machines. Experiments on Apache Spark’s distributed environment show that the proposed approach speeds up with the increase in number of machines and the notion of partition summary significantly reduces the amount of data shuffled among the machines.

Abstract

Community-driven Knowledge Sharing (KS) plat-forms have gained immense popularity in recent years among the Internet users to seek, learn & share information and expertise.These platforms encourage rich content by recognizing users’ contributions; measured as the reputation on platforms. Thus,peers on the platform dissuade other users to post low quality content by closing, disliking or not answering their questions.The aggressive attitude of the community leads to a negative impact on the users’ experience (especially for newbies) and the users tend to lose interest in the platform. Since the quality management on KS platforms is a necessity, we aim to emphasize the need of a mechanism to help users in improving their questions. To study this, we perform experiments on the Stack Overflow (SO) platform, however, the study can be adapted for other KS platforms.The SO community aggressively mark questions, even with slightest deformity, as closed. The non-triviality of reopening process and the rigorous review mechanism can be handled only by proper editing of the closed questions. Thus we present first-of-a-kind study, to assist the SO platform users, in the reopening process of their closed questions. We build predictive models to suggest the users that if their edited version of closed question will lead to a successful reopen or not. This can assist users at large by retracting them from entering the review process with improper edits. To learn these models effectively, we consider the user categories on the platform, based on their reputation: established& non-established. In addition to being the major contributor to closed questions, the non-established users have lower odds of getting their closed questions reopened than established users.Thus, we leverage the better editing skills of established users to learn question edit models by employing their reopened closed questions. We present the results of predictive model and suggest insights on the useful edits in the reopening process on the SO platform