US20050149383A1 - Method and system for ascribing a reputation to an entity as a rater of other entities - Google Patents
Method and system for ascribing a reputation to an entity as a rater of other entities Download PDFInfo
- Publication number
- US20050149383A1 US20050149383A1 US11/056,889 US5688905A US2005149383A1 US 20050149383 A1 US20050149383 A1 US 20050149383A1 US 5688905 A US5688905 A US 5688905A US 2005149383 A1 US2005149383 A1 US 2005149383A1
- Authority
- US
- United States
- Prior art keywords
- reputation
- entity
- rating
- ratee
- rater
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0201—Market modelling; Market analysis; Collecting market data
Definitions
- an electronic exchange is any exchange between two or more entities over an electronic network (i.e., not in person) such as, for example, a voice communications network (e.g., POTS or PBX) or a data communications network (e.g., LAN or the Internet) or a voice and data communications network (e.g., voice-over-IP network).
- Electronic exchanges may include electronic business transactions and electronic communications. Such electronic business transactions may include the negotiation and closing of a sale of goods or services, including solicitation of customers, making an offer and accepting an offer.
- consumer-to-consumer electronic marketplaces e.g., the eBay, OnSale, Yahoo and Amazon marketplaces found on the global Internet entities may transact for the sale and purchase of goods or services.
- Electronic communications also may include communications in on-line communities such as mailing lists, news groups, or web-based message boards and chatrooms, where a variety of sensitive personal information may be exchanged, including health-related data, financial investment data, help and advise on research and technology-related issues, or even information about political issues.
- an entity may be a person or an electronic agent (e.g., a software agent). Such a person may Act as an individual (i.e., on the person's own behalf) or as a representative (e.g., officer or agent) of a corporation, partnership, agency, organization, or other group.
- An electronic agent may act as an agent of an individual, corporation, partnership, agency, organization, or other group.
- an entity's identity may be anonymous to another entity. This anonymity raises several issues regarding trust and deception in connection to these exchanges. For example, an anonymous entity selling goods on-line may misrepresent the condition or worth of a good to a buyer without suffering a loss of reputation, business or other adverse effect, due to the entity's anonymity.
- a reputation mechanism is intended to provide an indication of how reliable an entity is, i.e., how truly its actions correspond to its representations, based on feedback by other entities that have conducted an electronic exchange with the entity. Such feedback typically is provided by another entity in the form of ratings in a numerical (e.g., 1-5) or Boolean (e.g., good or bad) form.
- a numerical e.g., 1-5
- Boolean e.g., good or bad
- Typical reputation mechanisms suffer from susceptibility to frauds or deceptions.
- a first typical fraud occurs when an anonymous entity, after developing a poor reputation over time in an on-line community, reenters the community with a new anonymous identity (i.e., on-line name), thereby starting anew with a higher reputation than the entity's already earned poor reputation.
- a second typical fraud to which typical reputation mechanisms are susceptible, occurs when two or more entities collude to provide high ratings for each other on a relatively frequent basis, such that the reputations of these entities are thereby artificially inflated.
- Sporas is a reputation mechanism for loosely-connected communities (i.e., one in which many entities may not have had an electronic exchange with one another and thus not have rated one another.)
- the acceleration factor is named as such because its value controls a rate at which an entity's reputation changes.
- the Sporas technique implements an entity reputation mechanism based on the following principles.
- new entities start with a minimum reputation value, and build-up their reputations as a result of their activities on the system. For example, if a reputation mechanism has a rating range from 1 to 100, then an entity may start with an initial reputation value, R 0 , of 1.
- R 0 initial reputation value
- Sporas reduces the incentive to, and effectively eliminates, that ability of an entity with a low reputation to improve the entity's reputation by reentering the system as a new anonymous identity.
- the reputations of each of the two or more entities involved are updated according to the feedback or ratings provided by the other entities, where the feedback or ratings represent the demonstrated trustworthiness of the two or more entities in the latest exchange. For example, referring to Equation 1 above, the ratee reputation R i of an entity is updated for each new rating, W i .
- two entities may rate each other only once. If two entities exchange more than once, then, for each entity, the reputation mechanism only applies the most recently submitted rating to determine the reputation of the rated entity. This fourth principle prevents two or more entities from fraudulently inflating their reputations, as describe above, by frequently rating each other with artificially high ratings.
- the reputation mechanism adapts to changes in an entity's behavior. For example, a reputation may be discounted over time so that the most recent ratings of an entity have more weight in determining the ratee reputation of the entity. For example, in Equation 1, above, ratings are discounted over time by limiting the effective number of ratings considered, C.
- the Sporas reputation mechanism also weights the reputation of a rated entity according to the reputation, R other , of another entity providing the rating, where this reputation of the other entity may be determined by applying Equation 1. Therefore, ratings from entities having relatively higher reputations have more of an impact on the reputation of the rated entity than ratings from entities having relatively lower reputations.
- Histos is a reputation mechanism better-suited for a highly-connected community, where entities have provided ratings for a significant number of the other entities. Histos determines a personalized reputation of a first entity from a perspective of a particular entity.
- Histos represents the principle that a person or entity is more likely to trust the opinion of another person or entity with whom she is familiar than trust the opinion of another person or entity who she does not know. Unlike Sporas, a reputation of first entity in Histos depends on the second entity from whose perspective the determination is made, and other ratings of the second entity provided by other users in an on-line community or population.
- FIG. 1 is a block diagram illustrating a representation of an on-line community or population 300 of entities A 1 -A 11 interconnected by several rating links, including rating links 302 , 303 , 304 , 306 , 308 and 310 .
- Each rating link indicates a rating of a rated entity (i.e., a ratee) by a rating entity (i.e., a rater) with an arrowhead pointing from the rating entity to the rated entity.
- a ratee is an entity in a position of being rated by one or more other entities
- a rater is an entity in a position of rating one or more other entities.
- rating link 302 represents a rating of 0.8 for ratee A 3 by rater A 1
- rating link 303 represents a rating of 0.9 for ratee A 1 by rater A 3 .
- each rating link only indicates a single rating, it is possible that an entity has provided more than one rating for another entity.
- the Zacharia reference discloses that if entity has provided more than rating for another entity, the most recent rating should be selected to determine a personalized reputation of a first entity from the perspective of a second entity.
- a first and second entity are connected if a rating path connects the first and second entity.
- a rating path is a series of rating links that connect a first entity to a second entity.
- entities A 1 and A 11 are connected by several rating paths, including rating paths 312 and 314 .
- Rating path 312 includes rating links 302 , 304 and 310
- rating path 314 includes rating links 302 , 306 and 308 .
- a breadth-first search algorithm is applied to find all of the rating paths connecting A 1 to A 11 that are of a length less than or equal to a specified value. If a rating link indicates more than one rating, then the most recent rating is selected for the determination of the personalized reputation.
- the number of rating links included in a rating path is referred to herein as the “length” of the rating path.
- an entity included along a rating path between the first rated entity and the second rating entity has a “level” equal to a number of links between the entity and the second entity.
- the entity A 8 is disposed along the rating path 314 .
- the entity A 8 has a level 2 in the context of the rating path 314 because two rating links 302 and 306 lie between the entity A 8 and the second entity A 1 .
- an entity having a level, L may be said to be a distance L away from the second entity.
- the Sporas reputation mechanism described in the Zacharia thesis uses a single reputation value to represent the reputation of an entity both as a ratee and a rater.
- This single reputation represents the reputation of an entity as a combination of ratings provided by other entities, but does not provide an indication of the reputation or trustworthiness of an entity as a rater of other entities.
- an entity may receive high ratings from other entities and thus have a high single-valued reputation, although the entity is a poor rater of other entities and thus should not truly have a high reputation as a rater of other entities.
- a “rater reputation” of an entity is a reputation or trustworthiness of the entity as a rater of one or more other entities
- a “ratee reputation” of an entity is the reputation of the entity according to the ratings of the entity provided by one or more other entities.
- Sporas does not provide a method or system for determining and maintaining a rater reputation of an entity.
- a method and system for determining a rater reputation of an entity Further, provided is a method and system for determining a ratee reputation of an entity based at least in part on the rater reputations of one or more of the entities that have rated the rated entity.
- a rater reputation of a first entity as a rater of other entities is determined.
- a first rating indicative of a rating of a rated entity by the first entity, and one or more second ratings, each second rating indicative of a rating of the rated entity provided by another entity, are provided.
- the second ratings are combined to produce a first combined rating.
- the first rating is compared to the first combined rating to produce a first rating predictability of the first rating, the first rating predictability being a negative function of a magnitude of a difference between the first rating and the first combined rating.
- a resulting rater reputation is generated based at least in part on the first rating predictability.
- This embodiment may be implemented as a computer program product that includes a computer readable medium and computer readable signals stored on the computer readable medium that define instructions. These instructions, as a result of being executed by a computer, instruct the computer to perform the Acts described above for this embodiment.
- a system for determining a rater reputation of a first entity as a rater of other entities is provided.
- a first rating indicative of a rating of a rated entity by the first entity, and one or more second ratings, each second rating indicative of a rating of the rated entity provided by another entity, is provided to the system.
- the system includes a rater reputation generator to combine the second ratings to produce a first combined rating.
- the rater reputation generator is further operative to compare the first rating to the first combined rating to produce a first rating predictability of the first rating, the first rating predictability being a negative function of a magnitude of a difference between the first rating and the first combined rating.
- the rater reputation generator is further operative to generate a resulting rater reputation based at least in part on the first rating predictability, and to output this generated rater reputation.
- a system for determining a rater reputation of a first entity as a rater of other entities is provided.
- a first rating indicative of a rating of a rated entity by the first entity, and one or more second ratings, each second rating indicative of a rating of the rated entity provided by another entity are provided for the system.
- the system includes: means for combining the second ratings to produce a first combined rating; means for comparing the first rating to the first combined rating to produce a first rating predictability of the first rating, the first rating predictability being a negative function of a magnitude of a difference between the first rating and the first combined rating; and means for generating a resulting rater reputation based at least in part on the first rating predictability.
- a ratee reputation of a first entity is determined.
- a first rating of the first entity by a second entity is received.
- One or more rater reputations including a first rater reputation of the second entity as a rater of other entities are accessed, and a ratee reputation of the first entity is generated by combining the one or more rater reputations and the first rating.
- This embodiment may be implemented as a computer program product that includes a computer readable medium and computer readable signals stored on the computer readable medium that define instructions. These instructions, as a result of being executed by a computer, instruct the computer to perform the Acts described above for this embodiment.
- a system for determining a ratee reputation of a first entity includes a ratee reputation generator to receive as input a first rating of the first entity by a second entity.
- the ratee reputation generator is operative to access one or more rater reputations including a first rater reputation of the second entity as a rater of other entities, and to generate a ratee reputation of the first entity by combining the one or more rater reputations and the first rating signal.
- the ratee reputation generator is further operative to provide as output the generated ratee reputation.
- a system for determining a ratee reputation of a first entity includes: means for receiving a first rating of the first entity by a second entity; means for accessing one or more rater reputations including a first rater reputation of the second entity as a rater of other entities; and means for generating a ratee reputation of the first entity by combining the one or more rater reputations and the first rating signal.
- FIG. 1 is a block diagram illustrating a representation of an on-line community of entities interconnected by several rating links;
- FIG. 2 is a flowchart illustrating an example embodiment of a method of determining a rater reputation of an entity
- FIG. 3 is a flow chart illustrating an example embodiment of a method of combining rating predictabilities to produce a rater reputation
- FIG. 4 is data flow diagram illustrating an example embodiment of a system for generating a rater reputation of an entity
- FIG. 5 is a data flow diagram illustrating an example embodiment of a system for generating a rater reputation of an entity
- FIG. 6 is a data flow diagram illustrating an example embodiment of a system for generating a ratee reputation deviation of an entity
- FIG. 7 is a data flow diagram illustrating an example embodiment of a system for generating a rater reputation deviation of an entity
- FIG. 8 is a flowchart illustrating an example embodiment of a method of generating a ratee reputation of an entity
- FIG. 9 is a data flow diagram illustrating an example embodiment of a system for generating a ratee reputation of an entity
- FIG. 10 is a flowchart illustrating an example embodiment of a method of determining a personalized ratee reputation
- FIG. 11 is a data flow diagram illustrating an example embodiment of a system for determining a personalized ratee reputation
- FIG. 12 is a flow chart illustrating an example embodiment of recursively estimating the impact of one or more ratings on a result
- FIG. 13 is a data flow diagram illustrating an example embodiment of a system for recursively estimating the impact of one or more ratings on a result
- FIG. 14 is a flowchart illustrating an example embodiment of a method of generating a weighting vector for a set of multiple ratings
- FIG. 15 is a data flow diagram illustrating an example embodiment of a system for generating a weighting vector for a set of multiple ratings
- FIG. 16 is a flow chart illustrating an example embodiment of a method of combining multiple estimated reputations to produce a new reputation
- FIG. 17 is a data flow diagram illustrating an example embodiment of a system for combining multiple estimated reputations to produce a new estimated reputation
- FIG. 18 is a data flow diagram illustrating an example system architecture for implementing the methods and systems of FIGS. 2-17 .
- determining a rater reputation is described below primarily in connection to ratings developed in connection with electronic exchanges, such determination may be applied to any of a variety of ratings, regardless of whether the rating is provided as a result of an electronic exchange.
- a rating may represent a qualitative assessment of an in-person interview of a job candidate, an in-person sale or a credit transaction.
- the rater reputation produced using the method and system described below may have any of a variety of applications such as, for example, assessing the trustworthiness (as a rater) of a reference, an employment recruiter, a rating agency, or any other entity that provides recommendations, scores, rankings or ratings of other entities.
- a rater reputation may be determined, for one or more ratings of one or more rated entities, respectively, provided by a first entity, by comparing each rating to other ratings of the rated entity provided by other entities. For each rated entity, the statistical similarity between the rating provided by the first entity and ratings provided by the other entities as determined by the comparison, i.e., the rating predictability, may serve as a basis for a rater reputation of the first entity.
- the ratings provided by the other entities may be ratings provided at points in time that occur after a point in time at which the rating by the first entity is provided. Determining a rater reputation of a first entity by comparing ratings provided by the first entity to such future ratings may provide a more accurate estimate of the first entity's rater reputation than comparing these ratings to past ratings of other entities. This more accurate estimate may result from the fact that if past ratings are used for the comparisons, and the first entity is aware that past ratings are being used, the first entity may access the past ratings and bias her ratings to be consistent with the past ratings, thus resulting in a higher reputation of the first entity than otherwise would occur.
- the first entity has less incentive to bias her ratings, resulting in more honest ratings being provided by the first entity. These more honest ratings result in a more accurate estimation of the rater reputation of the first entity.
- the result of the comparison of (a) the rating provided by the first rater of a rated entity and (b) other ratings of the rated entity provided by other raters may be weighed over a ratee reputation deviation of the rated entity, as will be described in more detail below in relation to FIG. 5 .
- This ratee reputation deviation represents a deviation of ratings of the rated entity from an expected value of the rating of the rated entity. Entities whose ratee reputations fluctuate over a wide range of values, such as new entities and entities that receive a wide range of ratings (i.e., unstable entities), typically have high ratee reputation deviations.
- This weighting of the comparison results in a rating predictability that is greater for greater values of ratee reputation deviation and less for lesser values of ratee reputation deviation.
- FIG. 2 is a flowchart illustrating an example embodiment of a method of generating a rater reputation of an entity, where a first rating of a rated entity by a first entity and second ratings of the rated entity provided by other entities are provided.
- the first and second ratings may be provided by accessing two or more entries of a data structure such as, for example, a database, stored on a computer-readable medium such as, for example, a non-volatile memory (e.g., a magnetic disk, CD ROM, or magnetic tape) or a volatile memory (e.g., an integrated semiconductor memory such as RAM).
- a non-volatile memory e.g., a magnetic disk, CD ROM, or magnetic tape
- a volatile memory e.g., an integrated semiconductor memory such as RAM
- Such a data structure may reside on a computer-readable medium located on a same computer at which an application running the method of generating resides and may be accessed by a bus or other known means.
- the data structure may reside at a remote location from such application, where access of the data structure may include use of one or more networks, bridges, routers, switches, hubs, other network devices, or any combination thereof.
- Such access may include wireless and wire (i.e., cable) transmissions.
- two or more of the second ratings of the rated entity may be combined to produce an expected rating.
- This expected rating represents an expected value of the first rating in consideration of the values of the two or more second ratings.
- the two or more ratings to be combined may be selected according to each rating's temporal proximity to the rating provided by the first entity (i.e., the first rating), or may be selected with other criteria such as, for example, demographic similarity of the entity that provided the second rating to the first entity.
- second ratings provided temporally after the first rating are selected.
- Such selected second ratings may be referred to herein as future ratings.
- a limiting factor may be provided to determine a number of the second ratings to select.
- the value of the limiting factor may be selected to achieve an acceptable balance between (a) rater reputation accuracy and (b) the time and cost of determining the rater reputation.
- second ratings provided by the first entity are excluded from selection. Such exclusion prevents the first entity from providing multiple ratings of the rated entity, and intentionally providing a same or similar value as each of the multiple ratings to artificially increase the first entity's rater reputation.
- the selected second ratings may be combined by calculating an average of the second ratings.
- this calculated average may be weighted, where each selected second rating is weighted according to the reputation of the entity that provided the second rating relative to the reputation of the other entities that provided second ratings.
- Equation 5 represents combining only future rating (from j+1 to k)
- other second ratings may be combined.
- 1/C is a change rate factor, described in more detail below in relation to Act 12 .
- a situation in which it may be desirable to determine an expected rating by combining selected second ratings other than future ratings is described below in more detail in relation to “seeding” a system.
- Weighting each second rating according to the reputation of the entity that provided the second rating generates an expected rating that gives more weight to ratings by those entities that have higher rater reputations. For example, if most of the higher second ratings were provided by entities having a low rater reputation, and most of the lower second ratings were provided by entities having a higher rater reputation, then the calculated expected rating would be lower than a raw average. This lower expected rating represents a bias towards the second ratings provided by the entities having a higher rater reputation.
- Such weighting of each selected second rating represents the principle that entities having higher rater reputations give more accurate ratings, and thus the ratings they provide are more valuable in determining a rater reputation of an entity (by definition) and, therefore, should be given more weight than ratings provided by entities having lower rater reputations.
- the average rating of the selected second ratings also may be weighted according to the relative proximity of the times at which the selected second ratings are provided to the time at which the first rating is provided. In other words, a rating provided closer in time to the first rating time will have a greater weight and thus more of an impact on the calculated average than a temporally more distant rating.
- This weighting reflects the principle that an entity's ratee reputation may change over time. Thus, for a given time at which a first rating is compared to other ratings, temporally closer ratings should be given more weight in determining the rater reputation of the first entity. Conversely, temporally more distant ratings should have lesser weight in determining the rater reputation of the first entity.
- time may be divided into a number of intervals, where each second ratings is provided during one of the intervals.
- Each second rating may be weighted according to the temporal proximity to the first point in time of the interval in which the second rating was provided.
- each selected second rating may be placed in a temporal order according to its relative temporal proximity to the first point in time, and each selected second rating may be weighted according to its position in this order. If only future ratings are selected, then the earlier the second rating is provided, the lower (i.e., the closer to the beginning) the rating is in the temporal order.
- the first rating of the rated entity, provided by the first entity may be compared to the expected rating, to produce a rating predictability.
- This rating predictability may be a negative function of an absolute difference between the first rating and the expected rating.
- a first value is a negative function of a second value if the first value decreases as a result of the second value increasing, and increases as a result of the second value decreasing.
- a first value is a positive function of a second value if the first value increases as a result of the second value increasing, and decreases as a result of the second value decreasing.
- comparing the first ratings and the second ratings may include determining the absolute difference (i.e., magnitude of the difference) between the first rating and the second rating and then applying a Gaussian distribution function to the determined absolute difference to produce the first rating predictability. If a Gaussian distribution function is being applied, then the absolute difference between the first rating and the expected rating may first be divided by a ratee reputation deviation of the rated entity.
- the ratee reputation deviation defines a deviation of ratings by other entities of the rated entity from an expected rating, and is described in more detail below in relation to FIG. 6 .
- Dividing the absolute difference by the rates reputation duration has the effect of weighting the absolute difference over the ratee reputation deviation. This weighting may be desirable to account for the stability or reliability of rater ratings of the rated entity in determining the rates reputation of the first entity. For example, if the ratee reputation of the rated entity is unstable (i.e., ratings provided for the rated entity vary considerably), the rater reputation of the first entity should be changed less by the difference between the expected rating and the first rating. Conversely, if the ratee reputation of the rated entity is stable, the rater reputation of the first entity should be changed more by the difference.
- Act 5 the other rating predictabilities may be combined with the rating predictability generated in Act 4 to produce the rater reputation of the first entity.
- Act 5 may be implemented in any of several different ways. In one implementation, Act 5 may be implemented by averaging all of the generated rating predictabilities associated with each rating provided by the first entity.
- Other methods may be used to determine a rating predictability of an entity.
- the generated rating predictability may alone serve as the rater reputation of the first entity.
- Act 5 the other rating predictabilities may be combined with the rating predictability generated in Act 4 to produce the rater reputation of the first entity.
- Act 5 may be implemented in any of several different ways. In one implementation, Act 5 may be implemented by averaging all of the generated rating predictabilities associated with each rating provided by the first entity.
- the average may be a weighted average.
- each rating predictability may be weighted as a positive function of the time at which the rating associated with the rating predictability is provided. For example, the earlier the associated rating is provided, and thus the further away in time from when the rater reputation is being determined, the lower the weighting of the predictability.
- each rating predictability may be placed in a temporal order according to the time at which its associated rating was provided, and each predictability may be weighted according its position in this order. Further, time may be divided into a number of intervals, and the temporal intervals may be placed in a temporal order. A rating predictability then may be weighted according to the position of the temporal interval in the temporal order.
- Weighting the rating as such represents the principle that an entity's rater reputation may change over time, and thus more recent ratings should be given more weight than older ratings in determining the rater reputation of an entity.
- Determining a rater reputation of a first entity by averaging rating predictabilities of the first entity may become more cost prohibitive as the number of ratings of other entities provided by the first entity increases.
- the rater reputation may be determined recursively. Determining a rater reputation recursively, for example, as described below, saves computational space and time, particularly as the number of ratings of other entities provided by the first entity grows.
- Such a recursive determination may include: providing a previously determined rater reputation (i.e., an initial rater reputation) of the first entity; determining a rater reputation adjustment (which may be a positive or negative value) based on the generated rating, and adding the reputation adjustments to the previously determined rater reputation to produce the rater reputation.
- a rater reputation adjustment which may be a positive or negative value
- FIG. 3 is a flow chart illustrating an example implementation of Act 5 .
- the rating predictability may be subtracted from the initial rater reputation to produce a reputation modification.
- the reputations of the entities that provided the selected second ratings may be combined to produce a combined reputation such as, for example, by calculating an average reputation from these reputations.
- Act 8 may be performed in sequence or concurrently with Acts 2 - 6 .
- the reputation modification may be scaled by the combined reputation to produce a scaled reputation modification.
- This scaling has the effect of weighting the reputation modification according to the reputations of the entities corresponding to the second ratings. If the combined reputation is an average reputation of the entities that provided the selected second ratings and has a relatively high value, then the reputation modification is scaled such that the reputation adjustment is relatively high. Conversely, if the average rater reputation of these raters is relatively low, then the reputation adjustment resulting from the scaled reputation modification is relatively low.
- the scaled reputation modification maybe divided by a change rate factor to produce a reputation adjustment.
- This change rate factor controls the rate at which the first rating and the resulting rating predictability can change the rater reputation of the first entity. For example, if the rate change factor has a high value, then the rate of change will be slower such that a relatively higher number of ratings will have to be provided by an entity before the rater reputation of the entity approaches an Actual reputation of the entity.
- the reputation adjustment is added to the initial rater reputation to produce a resulting rater reputation.
- the change rate factor is named as such because its value impacts the rate at which the rater reputation changes value.
- the value of C may be set equal to the value of T o , the effective number of observations, described below in relation to Equations 19 and 20.
- an original rater reputation i.e., an entity's rater reputation before a first rater reputation has been determined for the entity
- R 0 rater an entity's rater reputation has its lowest value before the entity provides a first rating of another entity.
- Such an original rater reputation prevents an entity with a low rater reputation from creating a new identity as a new entity and beginning with a higher rater reputation than the low reputation of the entity.
- Recursively determining a rater reputation of a first entity may result in an rater reputation that asymptotically approaches an Actual rater reputation of the first entity as the number or ratings of other entities provided by the first entity increases.
- An Actual rater reputation of the entity may be determined by averaging the determined rating predictabilities (and possibly weighting the average) for each rating provided by the first entity, for example, by applying Equation 11 or 12, or a variation thereof. As described above, this averaging may become cost prohibitive as the number of ratings provided by the first entity increases. Further, this averaging does not penalize an entity from starting over as a new entity by initializing to zero an original rater reputation of the entity. Both of these shortcomings of calculating an Actual reputation are avoided by initializing to zero an original rater reputation of an entity and by recursively determining the rater reputation of the entity, for example, by applying Equation 16 and variations thereof.
- the rater reputations of the entities that provided the most recent M ratings may be determined.
- each of the selected second ratings is one of the most recent M ratings.
- the amount that a rating entity's rater reputation is adjusted is proportional to the number of ratings received for a rated entity after the rating provided by the rating entity. This weighting reflects the principle that if there are more ratings received after the rating provided by the rating entity, then there is more information to estimate the predictability of the rating entity, as described above in relation to Equations 9 and 10, from which the rater reputation adjustment is determined. This use of more information results in more accurate estimations of the predictability of the rating entity, and a more accurate rater reputation adjustment. Accordingly, by weighting the rater reputation adjustment as described above in relation to Equation 17, the rater reputation adjustment is weighted as a function of this accuracy.
- the new entity may have an original rater reputation initialized to zero. Initializing an original rater reputation to zero may be desirable if an entity is capable of having multiple identities. In a system where an entity is restricted to a lifetime persistent identity, for example, a social security number, then the original rater reputation may not be initialized to zero, but may be initialized to another value. For example, the original rater reputation may be initialized to an average rater reputation value.
- an entire population of entities that have been using a rating system to rate each other, and thus have pre-collected ratings may want to determine a rater reputation of one or more, possibly all, entities of the population using these pre-collected ratings.
- Such determination of rater reputations for a population of entities using pre-collected ratings may be referred to herein as “seeding” a reputation system. If seeding a reputation system, one or more first entities of the population may not have provided a sufficient number of ratings of other entities such that a recursively generated rater reputation of these first entities approximates an Actual rater reputation of these first entities, respectively. Consequently, the resulting rater reputation of one or more first entities that have provided relatively few ratings is lower than the resulting reputation of one or more first entities that have provided relatively many ratings, irrespective of the Actual ratings provided by the first entities.
- pre-collected ratings may be used to determine the rater reputation as follows.
- a first pass of determining the rater reputation for example, by applying Equation 16 or variations thereof, the pre-collected ratings are used and the original rater reputation of the first new entity may be initialized to zero.
- a second pass of determining the rater reputation may use the same pre-collected ratings and initialize the original rater reputation of the first new user to the rater reputation resulting from the first pass. This initialization to a resulting reputation may be repeated in other passes until the resulting reputation adequately approximates the Actual rater reputation of the first new entity.
- FIG. 4 is a data flow diagram illustrating an example embodiment of a system 19 for generating a rater reputation 38 .
- the rater reputation generator 20 may receive a request 21 from a user indicating a request for a first entity's reputation. In response to the user request 21 , the rater reputation generator 20 may receive as input a first rater rating 26 and selected second ratings 28 , and generate the resulting rater reputation 38 as output, for example, by performing Acts 2 - 5 of FIG. 2 . In one implementation, the rater reputation generator 20 may also receive as input other rating predictabilities 49 to generate a resulting rater reputation 38 by averaging rating predictabilities, as described above in relation to FIG. 2 .
- the rater reputation generator 20 may also receive as input an initial rater reputation 22 , other rater reputations 24 , a ratee reputation deviation 30 , a range 32 , and a change rate factor 34 , and use these inputs in addition to inputs 26 and 28 to generate a resulting rater reputation 38 such as, for example, by applying Acts 6 - 14 of FIG. 3 .
- the rater reputation generator further may receive a value 31 indicating a number of most recent ratings and a value 33 indicating the number of ratings provided for the rated entity after the first rater rating 26 was provided.
- Values 31 and 33 correspond to values M and m, respectively described above in relation to Equation 17.
- FIG. 5 is a data flow diagram illustrating a more detailed example embodiment of the system 19 for generating a resulting rater reputation 38 .
- the rater reputation generator 20 may include an expected rating generator 40 , a predictability generator 44 , a reputation modification generator 48 , a combined predictability generator 47 , a combined reputation generator 52 , a reputation adjustment generator 56 and an adder 60 .
- the expected rating generator 40 may receive as input the selected second ratings 28 and generate as output an expected rating 42 .
- the expected rating generator 40 also may receive as input other rater reputations 24 , and use this input 24 in addition to input 28 , to generate the expected rating 42 .
- the other rater reputations 24 may be the reputations of the entities that provided the selected second ratings 28 .
- Expected rating generator 40 may generate expected ratings 42 in accordance with the various techniques described above in relation to Act 2 of FIG. 2 .
- the predictability generator 44 may receive as input the expected ratings 42 and the first rater rating 26 and produce a rating predictability 46 as output.
- the predictability generator 44 also may receive the ratee reputation deviation 30 and the range 32 , and use both of these inputs to help produce the rating predictability 46 .
- the predictability generator 44 may generate the rating predictability 46 in accordance with the various techniques discussed above in relation to Act 4 of FIG. 2 .
- the rater reputation of a first entity may be determined according to Equations 11 or 12 by including in the rater reputation generator the combined predictability generator 47 that receives as input rating predictability 46 and other rating predictabilities 49 (which may have been generated by predictability generator 44 ) and generates as output a resulting rater reputation 38 .
- the reputation modification generator 48 may receive a rating predictability 46 and an initial rater reputation 22 , and generate a reputation modification 50 .
- the reputation modification generator 48 may generate the reputation modification 50 by subtracting the rating predictability from the initial rater reputation as described above in relation to Act 6 of FIG. 3 .
- the combined reputation generator 52 may receive other rater reputations 24 and generate a combined reputation value 54 .
- the combined reputation generator 52 may calculate the average of the other rater reputations to determine the combined reputation 54 .
- a reputation adjustment generator 56 may receive the combined reputation 54 and the reputation modification 50 and generate a reputation adjustment 58 .
- the reputation adjustment generator 56 may also receive the change rate factor 34 .
- the reputation adjustment generator 56 may generate the reputation adjustment 58 in accordance with the techniques described above in relation to Acts 10 and 12 of FIG. 3 .
- the combined reputation 54 may scale the reputation modification 50 in accordance with the combined reputation 54 of the other raters.
- the reputation adjustment generator further receives values 31 and 33 corresponding to values M and m, respectively, and generates the rater reputation adjustment 58 as described above in relation to Equation 17.
- the adder 60 may add the initial rater reputation 22 to the reputation adjustment 58 to produce the resulting rater reputation 38 .
- the rater reputation generator 20 and any components thereof may be implemented as software, hardware, firmware or any combination thereof.
- the rater reputation generator 20 and any components thereof may reside on a single machine (e.g., a computer), or may be modular and reside on multiple interconnected (e.g., by a network) machines. Further, on each of the one or machines that include the rater reputation generator 20 or a component thereof, the generator 20 or component may reside in one or more locations on the machine. For example, different portions of the rater reputation generator 20 or different portions of a component may reside in different areas of memory (e.g., RAM, ROM, disk, etc.) on a computer.
- memory e.g., RAM, ROM, disk, etc.
- the range 32 and the change rate factor 34 may be constants for the system 19 and may be stored in a reputation database or other data structure, as described in more detail below in relation to FIG. 18 .
- the first rater rating 26 , the other reputations 24 , the selected second ratings 28 , the ratee reputation deviation 30 , the initial rater reputation 22 , and values 31 and 33 also may be stored in the reputation database or other structure.
- the rater reputation generator 20 may access the reputation database or similar data structure and retrieve values 26 , 32 , 24 , 28 , 26 , 30 , 31 , 32 , 33 and 34 to generate the resulting rater reputation 38 as described above.
- the resulting rater reputation 38 then may be stored in the reputation database or other data structure for later access.
- System 19 and the components thereof, are merely example embodiments of a system for generating a rater reputation. Such example embodiments are not meant to limit the scope of the invention and are provided merely for illustrative purposes, as any of a variety of other systems and components for determining a rater reputation, determining a may fall within the scope of the invention.
- FIG. 6 is a data flow diagram illustrating an example embodiment of a system 69 for recursively generating the ratee reputation deviation 30 of a rated entity.
- determining a ratee reputation deviation recursively saves computational space and time, particularly as the number of provided ratings for the rated entity grows.
- the ratee reputation deviation generator 72 may receive as input a ratee reputation 70 , a rater reputation 74 , an initial ratee reputation deviation 75 , a most recent rating 76 and a forgetting factor 77 .
- the ratee reputation deviation generator 72 may also receive the range 32 .
- the ratee reputation 70 is the ratee reputation determined from the most recent rating 76 of the rated entity, for example, by applying Equation 1 as disclosed in the Zacharia thesis, or by applying a variation of Equation 1, where the rater reputation is determined in accordance with Equation 15 or a variation thereof, as described above in relation to FIG. 2 .
- the initial ratee reputation deviation 75 is the ratee reputation deviation of the rated entity before the most recent rating 76 was provided.
- the rater reputation 74 is the rater reputation of an entity that provided the most recent rating 76 .
- the ratee reputation generator 72 may generate from these inputs the ratee reputation deviation 30 that represents a reliability of the rated entity's ratee reputation.
- the ratee reputation deviation generator 72 also may receive a minimum allowed rater reputation deviation and compare it to the determined rater reputation deviation 30 . If the minimum deviation is greater than deviation 30 , then the rater reputation deviation 30 may be set equal to the minimum allowed rater reputation.
- the expected rating may be calculated by applying Equation 2 as described in the Zacharia thesis.
- the forgetting factor is a constant having a value ⁇ 1 such that older ratings have less weight in determining the resulting ratee reputation deviation than more recent ratings.
- a forgetting factor may not be applied, or an average deviation may be determined for all determined ratee reputations of the rated entity, or such an average deviation may be calculated with each ratee reputation weighted according to how recent the ratee reputation was determined.
- an original ratee reputation deviation, RD i ratee is initialized to a maximum allowed value for a ratee reputation deviation.
- a maximum value may be predetermined. Consequently, a rated entity's ratee reputation deviation has its highest value before the rated entity is rated by other entities. Assigning a maximum value to an original ratee reputation deviation prevents an entity with a high ratee reputation deviation from creating a new identity as a new entity and beginning with a more desirable lower ratee reputation deviation.
- Equation 19 may be considered a recursive estimation algorithm of Recursive Least Squares (RLS) with a forgetting factor of F. Equation 19 estimates recursively an average square deviation of an actual rating from an expected (i.e., estimated) rating described in more detail below in relation to FIGS. 8 and 9 .
- RLS Recursive Least Squares
- Equation 19 estimates recursively an average square deviation of an actual rating from an expected (i.e., estimated) rating described in more detail below in relation to FIGS. 8 and 9 .
- Recursive Least Squares please refer to Chapter 9 of “Lecture Notes and Non-Linear and Non-Stationary Time Series Analysis,” by H. Madsen and J. Holst, Institute of Mathematical Modeling (IMM), Technical University of Denmark, Lyngby, Denmark, 1998 (hereinafter the Madsen text), the contents of which is herein incorporated by reference in its entirety.
- a forgetting factor may not be applied, or an average deviation may be determined for all determined ratee reputations of the rated entity, or such an average deviation may be calculated with each ratee reputation weighted according to how recent the ratee reputation was determined.
- the ratee reputation deviation generator 72 may be implemented as software, hardware, firmware or any combination thereof.
- the ratee reputation deviation generator 72 may reside on a single machine (e.g., a computer), or may be modular and reside on multiple interconnected (e.g., by a network) machines. Further, on each of the one or more machines that include the ratee reputation deviation generator 72 or modules thereof, the generator 72 or modules may reside in one or more locations on the machine. For example, different portions of the ratee reputation deviation generator 72 or modules may reside in different areas of memory (e.g., RAM, ROM, disk, etc.) on a computer.
- memory e.g., RAM, ROM, disk, etc.
- the range 32 and the forgetting factor 77 may be constants stored in a reputation database or similar data structure as described in more detail in relation to FIG. 18 .
- the ratee reputation 70 , the rater reputation 74 , and the initial ratee reputation deviation 75 also may be stored in the reputation database or similar structure.
- the ratee reputation deviation generator 72 in response to receiving the most recent rating 76 , may access the reputation database or similar structure to generate the ratee reputation deviation 30 as described above. The ratee reputation deviation 30 then may be stored in the reputation database or similar structure for later access.
- System 69 is merely an example embodiment of a system for generating a ratee reputation deviation. Such an example embodiment is not meant to limit the scope of the invention and is provided merely for illustrative purposes, as any of a variety of other systems for determining a ratee reputation deviation may fall within the scope of the invention.
- the ratee reputation deviation 30 it may be desirable to have an estimate of the reliability of the rater reputation of an entity that is providing a rating. Such a reliability may be estimated by calculating of a rater reputation deviation. Similarly to as described above in relation to ratee reputation deviation, computational space and time may be saved by calculating a rater reputation deviation recursively, particularly as the number of ratings of other entities provided by the first entity increases.
- Equation 21 estimate recursively an average square deviation of a rater reputation of an entity from an expected (i.e., estimated) rater reputation.
- the initial rater reputation deviation, R ⁇ ⁇ D j - 1 rater may be weighted by the forgetting factor, F, and the average square deviation may be divided by the effective number of determined reputations, T 0 .
- a forgetting factor may not be applied, or an average deviation may be determined for all determined rater reputations of the rating entity, or such an average deviation may be calculated with each rater reputation weighted according to how recent the rater reputation was determined.
- an original rater reputation deviation, RD j rater is initialized to a maximum value. This maximum value may be predetermined. Consequently, a first entity's rater reputation deviation has its highest value before the first entity provides a first rating of another entity. Assigning a maximum value for the rater reputation deviation prevents a first entity with a high rater reputation deviation from creating a new identity as a new entity and beginning with a lower rater reputation deviation.
- FIG. 7 is a data flow diagram illustrating an example embodiment of a system 79 for generating a rater reputation deviation.
- a rater reputation deviation generator 82 may receive as input an initial rater reputation 22 , a rating predictability 46 , an initial rater reputation deviation 80 and a forgetting factor 77 .
- Rater reputation deviation generator 82 also may receive other rater reputations 24 .
- the rater reputation deviation generator 82 may generate the rater reputation deviation 84 from inputs 22 , 24 , 46 and 80 . Alternatively, in place of inputs 22 , 24 and 46 , deviation generator 82 may receive the scaled reputation modification as input and generate the rater reputation deviation 84 in accordance with Equation 21 or variations thereof.
- the rater reputation deviation generator 82 may receive a minimum allowed rater reputation deviation, RD min rater , and compare it to the determined rater reputation deviation 84 . If the minimum deviation is greater than deviation 84 , then the rater reputation deviation 84 may be set equal to the minimum allowed rater reputation.
- the rater reputation deviation generator 82 may be implemented as software, hardware, firmware or any combination thereof.
- the rater reputation deviation generator 82 may reside on a single machine (e.g., a computer), or may be modular and reside on multiple interconnected (e.g., by a network) machines. Further, on each of the one or more machines that include the rater reputation deviation generator 82 or modules thereof, the generator 82 or modules thereof, the generator 82 or modules may reside in one or more locations on the machine. For example, different portions of the rater reputation deviation generator 82 may reside in different areas of memory (e.g., RAM, ROM, disk, etc.) on a computer.
- memory e.g., RAM, ROM, disk, etc.
- the forgetting factor 77 may be stored as a constant in a reputation database or similar data structure as described below in relation to FIG. 18 .
- the initial rater reputation 22 , the rating predictability 46 , the other rater reputations 24 and the initial rater reputation deviation 80 also may be stored in the reputation database or similar data structure.
- the rater reputation deviation generator 82 may access the reputation database or similar data structure to access and retrieve values 22 , 24 , 80 and 77 , and generate rater reputation deviation 84 .
- the rater reputation deviation 84 then may be stored in the reputation database or similar structure for later access.
- System 79 is merely an example embodiment of a system for generating a rater reputation deviation. Such an example embodiment is not meant to limit the scope of the invention and is provided merely for illustrative purposes, as any of a variety of other systems and components for determining a rater reputation deviation may fall within the scope of the invention.
- the rater reputation and the first rating may be used to determine, at least in part, a ratee reputation of the rated entity.
- a method and system for determining a ratee reputation of a rated entity based at least in part on one or more ratings of the rated entity provided by one or more entities and the rater reputations of these one or more entities.
- the ratee reputation of a rated entity is determined by averaging all of the ratings provided by other entities for the rated entity, and by weighting each rating with the rater reputation of the entity that provided the rating, to produce a weighted average.
- each rating may be further weighted with a positive function of the time at which the rating was provided. For example, the earlier the rating is provided, and thus the further away in time from when the ratee reputation is being determined, the lower the weighting of the rating.
- each rating may be placed in a temporal order according to the time at which the rating was provided, and each rating may be weighted according its position in this order. Further, time may be divided into a number of intervals, and the temporal intervals may be placed in a temporal order. A rating then may be weighted according to the position of the temporal interval in the temporal order.
- Other weightings and combinations of weightings may be applied to a rating to determine a ratee reputation of an entity.
- Determining a ratee reputation of an entity by averaging ratings may become more cost prohibitive as the number of ratings provided by other entities for the first entity increases.
- the ratee reputation may be determined recursively. Similarly to as described above in relation to determining a rater reputation, determining the ratee reputation recursively saves computational space and time, particularly as the number of ratings provided for the rated entity increases.
- a recursive determination may include: providing a previously determined ratee reputation (i.e., an initial ratee reputation) of the entity; determining a ratee reputation adjustment (which may be a positive or negative value) based on a received rating, and adding the reputation adjustments to the previously determined rater reputation.
- a ratee reputation adjustment which may be a positive or negative value
- FIG. 8 is a flow chart illustrating an example implementation of a method of recursively determining a ratee reputation of an entity.
- an expected rating may be determined from the range and the ratee's initial reputation.
- An expected rating may be determined by applying Equation 4 or a variation thereof.
- the expected rating may be subtracted from the first rating to produce a ratee reputation modification.
- the ratee reputation modification may be produced as described in relation to Equation 23 below.
- the ratee reputation modification may be scaled by a rater reputation of the rating entity to produce a scaled ratee reputation modification.
- This rater reputation may be the resulting rater reputation generated according to one of the methods described in relation to FIGS. 2 and 3 , or a variation thereof.
- the scaled ratee reputation modification may be damped by a damping factor to produce a damped ratee reputation modification.
- the damping factor may be determined (in series or concurrently with Acts 90 - 94 ) by applying Equation 2, where the acceleration factor, a, may be predetermined to control the rate at which the ratee reputation can change.
- the damped ratee reputation modification may be divided by a change rate factor to produce a ratee reputation adjustment.
- the change rate factor affects the rate at which the ratee reputation may change.
- the ratee reputation may be added to the initial ratee reputation to produce a resulting ratee reputation.
- an original ratee reputation, R 0 rater for the entity has its lowest value before any ratings are provided for the entity. Assigning a lowest value to an original ratee reputation prevents an entity with a low ratee reputation from creating a new identity as a new entity and beginning with a higher ratee reputation.
- a ratee reputation of a first entity may be used for a variety of purposes, including determining whether to transact with the first entity, determining a price to pay for a good or service of the first entity and determining a price to pay for insuring a quality of a good or service of the first entity.
- FIG. 9 is a data flow diagram illustrating an example embodiment of a system 109 for generating a resulting ratee reputation 140 .
- a ratee reputation generator 110 may receive an acceleration factor 112 , a range 32 , an initial ratee reputation 114 , a first rating 116 , a rater reputation 118 and a change rate factor 120 , and generate a resulting ratee reputation 140 as output.
- the ratee reputation generator 110 may include a damping factor generator 130 , an expected rating generator 122 , a ratee reputation modification generator 126 , a ratee reputation adjustment generator 134 and an adder 138 .
- the expected rating generator 122 may receive as input the range 32 and the initial ratee reputation 114 , and generate an expected rating 124 , for example, by applying Equation 3.
- the ratee reputation modification generator 126 may receive the first rating 116 and the expected rating 124 , and generate the ratee reputation modification 128 , for example, as described above in relation to Equation 23.
- the damping factor generator 130 may receive the acceleration factor 112 , the initial ratee reputation 114 and the range 32 , and generate the damping factor 132 , for example, by applying Equation 3 above.
- the ratee reputation adjustment generator 134 may receive the ratee reputation modification 128 , the damping factor 132 , the rater reputation 118 and the change rate factor 120 and generate the ratee reputation adjustment 136 , for example, by applying Equation 25 above.
- the adder 138 may receive the initial ratee reputation 114 and the ratee reputation adjustment 136 and generate the resulting ratee reputation 140 , for example, by applying Equation 26 above.
- the system 109 also may include a transaction module such as transaction module 528 described below in relation to FIG. 11 .
- the transaction module may receive the resulting ratee reputation 140 , determine whether to transact with the rated entity based on the resulting ratee reputation, and then output a value, for example, a boolean value, which indicates whether or not to transact with the rated entity.
- the ratee reputation generator 110 and any combination of its components 110 , 122 , 126 and 134 may be implemented using software, firmware or hardware, or any combination thereof.
- the ratee reputation generator 110 and any components thereof may reside on a single machine (e.g., a computer), or may be modular and reside on multiple interconnected (e.g., by a network) machines. Further, on each of the one or machines that include the ratee reputation generator 110 or a component thereof, the generator 110 or component may reside in one or more locations on the machine. For example, different portions of the ratee reputation generator 110 or different portions of a component may reside in different areas of memory (e.g., RAM, ROM, disk, etc.) on a computer.
- memory e.g., RAM, ROM, disk, etc.
- the acceleration factor 112 , the range 32 and the change rate factor 120 may be stored as constants in a reputation database or similar data structure as described below in relation to FIG. 18 .
- the initial ratee reputation 114 and the rater reputation 118 may also be stored in the reputation database or similar data structure.
- the ratee reputation generator may access the reputation database and retrieve values 112 , 32 , 114 , 116 , 118 and 120 to generate the resulting ratee reputation 140 as described above.
- the resulting ratee reputation 140 then may be stored in the reputation database or similar structure for later access.
- System 109 and components thereof, are merely example embodiments of a system for generating a ratee reputation. Such example embodiments are not meant to limit the scope of the invention and are provided merely for illustrative purposes, as any of a variety of other systems and components for determining a ratee reputation may fall within the scope of the invention.
- FIG. 10 is a flowchart illustrating an example embodiment of a method for determining a personalized ratee reputation.
- a breadth-first search is performed beginning at the second entity to determine, from one or more rating paths, one or more first rating paths that have a first length.
- a next Act 404 for each determined first rating path, a third entity that has a level equal to or less than the first length is identified.
- a first rating of the first identity provided by the third entity is determined. For each identified third entity, it may be determined that the third entity has provided more than one rating of the first entity. For each third entity for which it has been determined that more than rating has been provided, a most recent rating may be selected from the one or more ratings to serve as the first rating of the first entity provided by the third entity.
- the first ratings are combined.
- the first ratings may be combined in any of a variety of ways.
- an average of the first ratings is calculated. Calculating this first average may include, for each first rating, weighting the first rating as a function of a personalized ratee reputation of the corresponding third entity from the prospective of the second entity. This weighting may be relative to personalized ratee reputations of the other third entities from the perspective of the second entity.
- the personalized ratee reputation of the third entity may be determined in any of a variety of ways.
- one or more fourth entities that are on the first rating paths, that have provided a second rating of the third entity and that have a level equal to one less than the level of the third entity are determined.
- the second ratings provided by these one or more fourth entities are then combined to produce the personalized ratee reputation of the third entity from the perspective of the second entity.
- Combining the second ratings may include calculating an average of the second ratings.
- Equation 4 or a variation thereof may be applied to combine the ratings.
- the personalized ratee reputation is produced by weighting the combined first rating as a function of the first length.
- This personalized ratee reputation may then be used to determine whether to transact with the first entity.
- a personalized ratee reputation of a first entity may be used for a variety of purposes, including determining whether to transact with the first entity, determining a price to pay for a good or service of the first entity and determining a price to pay for insuring a quality of a good or service of the first entity.
- FIG. 11 is a data flow diagram illustrating an example embodiment of a system 500 for determining a personalized ratee reputation of a first entity from the perspective of a second entity.
- the system 500 may include a personalized ratee reputation generator 506 and a transaction module 528 .
- the personalized ratee reputation generator 506 may include a path-searching module 508 , a first look-up module 514 , a processing module 512 , a second look-up module 520 and a ratings combiner 524 .
- the path-searching module 508 may receive an indication of the first entity 502 , an indication of the second entity 504 and other ratings 517 , and generate a signal indicating first rating paths 510 .
- the path-searching module may determine the first rating paths 510 as described above in relation to Act 402 of FIG. 10 .
- the path-searching module 508 may determine the first rating path 510 by searching or looking-up rating values in a ratings database or reputation database as described in more detail below in connection to FIG. 18 .
- the first look-up module 514 may receive the indication of the first rating paths 510 and generate third entities 518 , for example, as described above in connection to Act 404 .
- the first look-up module 514 may determine the third entities 518 by looking up the third entities in a ratings or reputation database as described in more detail in relation to FIG. 18 .
- the second look-up module 520 may receive the indication of the third entity 518 as input and generate as output the first ratings 522 , for example, as described above in relation to Act 406 .
- the second look-up module 520 may determine the first ratings 522 by looking up the first ratings in a ratings or reputation database, using the third entity 518 as a look-up criteria, as described in more detail below in relation to FIG. 14 .
- the processing module 512 receives the indication of the first rating paths 510 and the indication of the third entities 518 and generates second ratings 516 .
- the processing module 512 may generate the second ratings 516 as described above in relation to Act 410 .
- the processing module 512 may determine the second ratings 516 by looking up values in the ratings or reputation database described below in relation to FIG. 14 , using the first rating paths 510 and the third entities 518 as search criteria.
- the ratings combiner 524 receives the first ratings 522 and the second ratings 516 and generates the personalized ratee reputation 526 .
- the ratings combiner 524 may also receive other ratings 517 , which may have been generated in a similar manner to the first ratings 522 and the second ratings 516 , for example, by applying Equation 27.
- the ratings combiner 524 may generate the personalized ratee reputation 526 as described above in relation to Act 410 , and then may store the personalized ratee reputation in the reputation database or similar structure for later access.
- the transaction module 528 may receive the personalized ratee reputation 526 , determine whether to transact with the first entity based on the personalized ratee reputation, and then output a value, for example, a Boolean value 530 , which indicates whether or not to transact with the first entity.
- the boolean rating 530 then may be stored in the reputation database or similar structure for later access.
- the transaction module 528 , the personalized reputation generator 506 and any component thereof, including the path searching module 508 , the first lookup module 514 , the second lookup module 520 , the processing module 512 and the ratings combiner 524 , may be implemented as software, hardware, firmware or any combination thereof.
- the module 528 , generator 506 and components thereof may reside on a single machine (e.g., a computer), or may be modular and reside on multiple interconnected (e.g., by a network) machines. Further, on each of the one or more machines that include the transaction module 528 , personalized reputation generator 506 or a component thereof, the generator 506 , transaction module 528 or component may reside in one or more locations on the machine. For example, different portions of the personalized reputation generator 506 , different portions of a component, and different portions of the transaction module 528 may reside in different areas of memory (e.g., RAM, ROM, disk, etc.) on a computer.
- memory e.g., RAM, ROM, disk,
- System 500 and components thereof, are merely example embodiments of a system for generating a personalized reputation generator and transacting based on the personalized reputation generator 506 .
- Such example embodiments are not meant to limit the scope of the invention and are provided merely for illustrative purposes, as any of a variety of other systems and components for generating a personalized ratee reputation and transacting based thereon may fall within the scope of the invention.
- the rating entity may provide a rating for the quality of the good, a rating for the service quality, a rating for the promptness of delivery and an overall rating for the exchange. If such multiple ratings are received for a single exchange, the systems and methods described above for determining reputations and deviations may be applied to each rating independently. For example, the rater reputation and rater reputation deviation of a rating entity as a rater of the quality of the good may be determined or the ratee reputation and ratee reputation deviation of a rated entity for the promptness of its delivery of goods may be determined.
- a method and system for recursively estimating the impact of one or more attributes, e.g., ratings, on a result corresponding to a qualitative assessment, e.g., an overall rating, where the qualitative assessment is based on the one or more attributes will now be described. Determining such a recursive estimate over time allows the impacts of one or more ratings to be incrementally corrected as more ratings are received.
- attributes e.g., ratings
- a qualitative assessment e.g., an overall rating
- one or more objective metrics may be assigned according to the quality of a good, the time of delivery, the cost of the good, etc., and the impact of this metric on a result or qualitative assessment may be determined.
- a result of a qualitative assessment may be the qualitative assessment itself such as, for example, feedback, a rating or a score, or may be an action taken as a result of a qualitative assessment such as setting a price or making a selection. For example, if several entities purchase a good from a seller, and each entity provides separate ratings for the quality of the good, the service quality and the promptness of delivery, and also provides an overall rating of the seller, it may be desirable to determine the impact of each rating on the overall rating. Subsequently, the impact of each rating may be applied to estimate the impact of future ratings on a future overall rating.
- c) Determining a frequency (i.e., a selectivity ratio) that a particular entity, item or service is selected when compared with other entities items or services;
- FIG. 12 is a flow chart illustrating an example embodiment of a method 800 for estimating the impacts of a plurality of ratings on a result.
- a plurality of ratings provided for a first entity by a second entity in association with an exchange between the first and second entity are received.
- Each of these ratings corresponds to a respective rating parameter for the first entity.
- rating parameters may include quality of service, speed of delivery and quality of a good, and the corresponding ratings may be 0.5, 0.7 and 0.2, respectively.
- a result based on the exchange between the first and second entity may be received.
- This result may correspond to a result parameter for the first entity.
- the result parameter may be an overall rating or a price, and the corresponding result may be 0.7 or $7.95, respectively.
- a corresponding initial weighting value that corresponds to the rating parameter of the received rating is accessed.
- This initial weighting represents an estimated weight of the received rating for the rating parameter relative to received ratings for the other rating parameters in determining the result for the result parameter.
- the received ratings for the quality of a good sold by a first entity may have more weight in determining a price of the good than ratings received for the speed of delivery.
- the speed of delivery may have a corresponding initial weighting value of 0.6
- the quality of the good may have a corresponding initial weighting value of 0.4.
- the initial weighting value may be adjusted as a function of the weighting modification to produce a resulting weighting value for the corresponding rating parameter.
- the weighting modification may be added to each initial weighting value.
- the initial weighting values may be adjusted as a function of previous ratings provided for the rating parameters, for example, as described in more detail below in connection to FIG. 14 .
- FIG. 13 is a data flow diagram illustrating an example embodiment of a system for estimating the impacts of a plurality of ratings on a result.
- the system 169 may include a ratings database 205 and a weightings generator 170 that includes a weighting modification generator 194 and a weighting adjuster 195 .
- the ratings database 205 may be a reputation database or similar data structure such as described below in relation to FIG. 18 .
- the weighting modification generator 194 may receive ratings 175 and a result 178 , access initial weighting values 179 corresponding to the ratings 175 , and generate the weighting modification 196 , as described above in Acts 802 - 810 of FIG. 12 .
- the ratings 175 may be received along with a first entity ID 177 that identifies the first entity for which the ratings 175 are being provided.
- the weighting modification generator 194 may access the initial weighting values 179 from the ratings database 205 using the first entity ID 177 .
- the weighting modification generator 194 may generate the weighting modification 196 as described above in relation to Act 156 of FIG. 14 .
- the weighting adjuster 195 may receive the weighting modification 196 and the initial weighting values 179 , and generate the resulting weighting values 203 as described above in relation to Act 812 . In an embodiment, the weighting adjuster 195 may generate the resulting weighting values as described below in relation to Acts 158 and 160 of FIG. 14 .
- the weighting adjuster 194 may store the resulting weighting values 203 and the ratings database 205 for later access.
- the initial weighting values and resulting weighting values may be represented as weighting vectors or weighting matrices.
- the plurality of provided ratings may be represented by a rating vector or rating matrix, as will be described in more detail below.
- Such vectors and matrices may be embodied tangibly on a medium using any of a variety of data structures, for example, an array. Accordingly, the terms “weighting array” and “rating array,” as used herein, refer to a weighting vector or matrix and a rating vector or matrix, respectively, tangibly embodied on a medium as an array of other data structure.
- FIG. 14 is a flow chart illustrating an example embodiment of a method of estimating the impacts of a plurality of ratings on a result using vectors.
- Act 150 a multi-rating vector is transposed to produce a transposed multi-rating vector.
- Each value of the multi-rating vector may represent an attribute or sub-rating of an overall rating or other qualitative assessment.
- a resulting adjustment matrix may be generated from an initial adjustment matrix, the multi-rating vector and the transposed multi-rating vector.
- the initial adjustment matrix may be weighted by a forgetting factor, F, similar to as described above in relation to Equations 18 and 19.
- the forgetting factor, F may have the affect of giving the multi-rating vector more weight than the initial adjustment matrix in determining the resulting adjustment matrix.
- the forgetting factor may be chosen such that the initial adjustment matrix has more weight in determining the resulting adjustment matrix than the multi-rating matrix.
- the resulting adjustment matrix may be inverted to produce an inverted adjustment matrix.
- a weighting modification may be generated from the transposed multi-rating vector, an initial weighting vector and a result (e.g., an overall rating or other qualitative assessment).
- the weighting modification represents a difference between the result Y t and an estimated result, X t T Z t ⁇ 1 , according to the initial weighting vector and the multi-rating vector.
- a weighting adjustment vector may be generated from the inverted adjustment matrix, the multi-rating vector and the weighting modification.
- the weighting adjustment, Z adj further may be weighted as a function of any number of factors.
- the weighting adjustment may be weighted as a function of the reputation of the rater that provided the multi-rating vector and overall result, or as a function of some other factor corresponding to the transaction on which the multi-rating vector and the overall result are based, e.g., a factor indicative of the significance of the transaction.
- the weighting adjustment vector may be added to the initial weighting vector to produce a resulting weighting vector.
- each weighting vector includes an extra weighting value such that if there is a number of rating parameters n, then the dimension of the weighting vector is n+1. This extra weighting value may be used to factor in a parameter other than a rating parameter in determining a result parameter.
- estimated impacts of a plurality of ratings are determined for a single result.
- weighting vectors are applied to rating vectors and compared to a single scalar result.
- more than one result may correspond to a plurality of ratings. If more than one result is involved, the plurality of results may be represented with a vector, and the impacts of the plurality of ratings on the plurality of results may be represented by a weighting matrix, as opposed to a weighting vector.
- weighting matrices and a result vector as opposed to weighting vectors and a result scalar are intended to fall within the scope of the invention. Weighting vectors and result scalars are described for illustrative purposes and are not intended to limit the scope of the invention.
- a estimated result deviation may be determined.
- the estimated result deviation may be determined recursively.
- RD i weight is the resulting estimated result deviation
- RD i - 1 weight is an initial estimated result deviation
- F is a forgetting factor similar to as described above in relation to Equations 18 and 19
- T o is the effective number of estimated results similar to as described above in relation to Equations 18 and 19.
- FIG. 15 is a data flow diagram illustrating an example embodiment of the system 169 that generates a resulting weighting vector 204 to represent the estimated weights of a plurality of ratings in determining a result.
- the weighting generator 170 may receive as input an initial adjustment matrix 174 , a multi-rating matrix 176 , an overall rating (or other qualitative assessment) 178 and an initial weighting vector 180 , and generate as output a resulting weighting vector 204 .
- the weighting generator 170 may include an adjustment matrix generator 182 , an inverse matrix generator 190 , a transposed vector generator 186 , the weighting modification generator 194 and the weighting adjuster 195 .
- the weighting adjuster 195 includes a weighting adjustment generator 198 and a vector adder 202 .
- the transposed vector generator 186 may receive as input the multi-rating vector 176 and generate the transposed multi-rating vector 184 as output.
- the adjustment matrix generator 182 may receive as input the transposed multi-rating vector 184 and the initial adjustment matrix 174 and generate as output a resulting adjustment matrix 188 .
- the adjustment matrix generator 182 may also receive the forgetting factor 172 as input and generate the resulting adjustment matrix 188 based at least in part on the forgetting factor 172 .
- the resulting adjustment matrix 188 may be generated by applying Equation 29.
- the inverse matrix generator 190 may receive as input a resulting adjustment matrix 188 and generate as output an inverse adjustment matrix 192 .
- the weighting modification generator 194 may receive as input the transposed multi-rating vector 184 and the overall rating 178 and generate as output a weighting modification 196 .
- the weighting modification generator 194 may generate the weighting modification in accordance with Equation 30 above.
- the weighting adjustment generator 198 may receive as input the weighting modification 196 , the multi-rating vector 176 and the inverse adjustment matrix 192 , and generate as output the weighting adjustment vector 200 .
- the weighting adjustment generator 198 may generate the weighting adjustment vector 200 in accordance with Equation 31 above.
- the vector adder 202 may receive as input the weighting adjustment vector 200 and the initial weighting vector 180 and generate as output the resulting weighting vector 204 .
- the vector adder 202 may generate the resulting weighting vector 204 in accordance with Equation 32.
- the weighting generator 170 and any components thereof may be implemented as software, hardware, firmware or any combination thereof.
- the weighting vector generator 170 and any components thereof may reside on a single machine (e.g., a computer), or may be modular and reside on multiple interconnected (e.g., by a network) machines. Further, on each of the one or more machines that include the weighting vector generator 169 or a component thereof, the generator 170 or component may reside in one or more locations on the machine. For example, different portions of the weighting vector generator 170 or different portions of a component may reside in different areas of memory (e.g., RAM, ROM, disk, etc.) on a computer.
- memory e.g., RAM, ROM, disk, etc.
- the system 169 and components thereof are merely example embodiments of a system for generating a ratee reputation. Such example embodiments are not meant to limit the scope of the invention and are provided merely for illustrative purposes, as any of a variety of other systems and components for estimating the impacts of a plurality of ratings on a result may fall within the scope of the invention.
- any number of entities may be members of a population. This population may be grouped into clusters having common properties, for example, similar demographics or preferences. Further, for a transaction having multiple parameters, for example, multiple rating parameters, the entities of the population may be grouped into any number of clusters based on the statistical similarity of the ratings they provide for the parameters.
- the entities may be grouped in to clusters using any of a number of known clustering algorithms, for example, a k-means clustering algorithm, an example of which is described at: http://www.cne.gmu.edu/modules/dau/stat/clustgalgs/clust5_bdy.html.
- reputations, estimators, deviations, and other values may be determined for each cluster separately. For example, weighting vectors and associated matrices may be generated and maintained independently for each cluster.
- determining an estimated ratee reputation of a first entity using the rater reputations of one or more entities that provided ratings for the first entity is described above in relation to FIGS. 8 and 9 .
- an estimated ratee reputation may be determined for each attribute.
- an estimated ratee attribute reputation is an estimated ratee reputation determined for one of multiple attributes for a single exchange, as opposed to an estimated ratee reputation determined for a result or qualitative assessment associated with the single exchange.
- initial weighting values may be applied to associated attributes to estimate a ratee reputation of an entity.
- a method and system for estimating a ratee reputation of a first entity based on multiple estimated ratee reputations of the first entity may be desirable to combine two or more estimated ratee reputations to produce a new estimated ratee reputation.
- the deviation of such a new estimated ratee reputation over time from actual received ratings may be used to determine whether the new estimated ratee reputation is more accurate than the two or more estimated ratee reputations. Accordingly, provided herein is a method and system for estimating a ratee reputation of a first entity based on multiple estimated ratee reputations of the first entity.
- FIG. 16 is a flowchart illustrating an example embodiment of a method 600 of combining a first estimated ratee reputation of a first entity with a second estimated ratee reputation of the first entity to produce a third estimated ratee reputation.
- a reputation database for example, the reputation database 726 described below in relation to FIG. 17 , may store a first estimated ratee reputation and one or more ratee attribute reputations.
- the first estimated ratee reputation may have been generated by combining a plurality of overall ratings, where each overall rating was provided by a different entity for a corresponding transaction.
- the first estimated ratee reputation may be determined from the plurality of overall ratings in a similar manner to that described above in relation to FIGS. 8 and 9 .
- Each of the one or more ratee attribute reputations may have been generated by combining a plurality of attributes, where each attribute corresponds to a different one of the transactions. For example, each of the one or more ratee attribute reputations may be determined from the plurality of attributes in a similar manner to that described above in relation to FIGS. 8 and 9 .
- the one or more ratee attribute reputations are received.
- a corresponding weighting value is accessed.
- the accessed weighting value represents an estimated weight of the received ratee attribute reputation relative to others of the one or more received ratee attribute reputations in estimating a ratee reputation of the first entity, similarly to as described above in relation to Act 806 of FIG. 12 .
- a second estimated ratee reputation of the first entity is determined by applying each weighting value to its corresponding ratee attribute reputation.
- the weighting values may be combined with the corresponding ratee attribute reputations similarly to as described above in relation to Act 808 of FIG. 12 and Act 156 of FIG. 14 .
- the first estimated ratee reputation is accessed, for example, from a database. Act 608 may be performed serially or concurrently to the sequence of Acts 602 - 606 .
- the first estimated ratee reputation and the second estimated ratee reputation may be combined to produce a third estimated ratee reputation.
- the average of the first estimated ratee reputation and the second estimated ratee reputation may be determined such that the third estimated ratee reputation is an average of the first and second estimated ratee reputations.
- the first and second estimated ratee reputations may be weighted by estimated ratee reputation deviations to calculate a weighted average.
- RD 1 rater is a first estimated ratee reputation deviation corresponding to the first estimated ratee reputation and may be determined by application of Equation 21 as described above.
- RD 2 rater is a second estimated ratee reputation deviation corresponding to the second estimated ratee reputation and may be determined by application of Equation 34 or 35 as described above.
- a higher estimated ratee reputation deviation represents a lower reliability of the estimated ratee reputation and, conversely, a lower estimated ratee reputation deviation represents a higher reliability of the estimated ratee reputation. Therefore, if both the first and second estimated ratee reputations were weighted according to their respective ratee reputation deviations, the ratee reputation with a higher deviation and lower reliability would be given more weight (i.e., have a greater impact) in generating the third estimated ratee reputation, which consequently would generate a less reliable third estimated ratee reputation than that defined by Equation 36.
- Equation 36 to weight both the first and second ratee reputations according to their respective reliabilities, the first ratee reputation is weighted by the second ratee reputation deviation, and the second ratee reputation is weighted by the first ratee reputation deviation.
- the ratee reputation having the higher reliability will have the greater impact on the third estimated ratee reputation.
- FIG. 17 is a data flow diagram illustrating an example embodiment of a system 700 for generating an estimated ratee reputation.
- the system 700 may include a reputation database 726 and an estimated ratee reputation generator 706 that includes a first ratee reputation estimator 708 and a second ratee reputation estimator 712 .
- the reputation database 726 may be a reputation database or similar data structure as described below in relation to FIG. 18 .
- the estimated ratee reputation generator 706 may receive ratee attribute reputations 702 and a ratee ID 704 .
- the ratee attribute reputations 702 may be determined, as described above, from attributes corresponding to transactions with a first entity, and the ratee ID 704 may indicate the first entity.
- the estimated ratee reputation generator 706 may use the ratee ID 704 to access, from reputation database 726 , weighting values 718 , first estimated ratee reputation deviation 720 , second estimated ratee reputation 722 and second estimated ratee reputation deviation 724 .
- the estimated ratee reputation generator 706 may use values 718 , 720 , 722 and 724 to generate a third estimated ratee reputation 714 , for example, as described above in relation to FIG. 16 .
- the first ratee reputation estimator 708 may receive the one or more ratee attribute reputations 702 and the weighting values 718 , and generate the second estimated ratee reputation 710 , for example, as described above in relation to Act 606 of the FIG. 16 .
- the second ratee reputation estimator 712 may receive the first estimated ratee reputation deviation 720 , the second estimated ratee reputation 722 , the second estimated ratee reputation deviation 724 and the second estimated ratee reputation 710 , and generate the third estimated ratee reputation 714 , for example, as described above in relation to Act 610 of FIG. 16 .
- the estimated ratee reputation generator 706 and any of a combination of its components, including first ratee reputation estimator 708 and second ratee reputation estimator 712 , may be implemented using software, firmware or hardware or any combination thereof.
- the estimated ratee reputation generator 706 and any components thereof may reside on a single machine (e.g., a computer), or may be modular and reside on a multiple interconnected (e.g., by a network) machines. Further, on each of the one or more machines that include the estimated ratee reputation generator 706 or a component thereof, the generator 706 or component may reside in one or more locations on the machine. For example, different portions of the estimated ratee reputation generator 706 or different portions of a component may reside in different areas of memory (e.g., RAM, ROM, disk, etc.) on a computer.
- memory e.g., RAM, ROM, disk, etc.
- System 700 and components thereof, are merely example embodiments of a system for estimating a ratee reputation. Such example embodiments are not meant to limit the scope of the invention and are provided merely for illustrative purposes, as any of a variety of other system and components for estimating a ratee reputation may fall within the scope of the invention.
- FIG. 18 is a data flow diagram illustrating an example system architecture 209 for implementing the methods, systems and variations thereof described above in relation to FIGS. 2-17 .
- the system 209 may include a client 210 , a server 212 , ratee reputation database 234 , an authentication I.D. database 236 and a rater reputation database 238 .
- the components 210 , 212 , 234 , 236 and 238 of the system 209 may have a variety of configurations. For example, all these components may reside on a single computer, or any combination thereof may reside on a separate computer or multiple computers interconnected, for example, by a network. Further, any combination of these components may reside on separate networks, including separate LANs (Local Area Networks), MANs (Metropolitan Area Networks) and WANs (Wide Area Networks).
- LANs Local Area Networks
- MANs Micropolitan Area Networks
- WANs Wide Area Networks
- the ratee reputation database 234 may contain one or more entries, where each entry represents ratee reputation data for an entity.
- an entry in the ratee reputation database may include an entity I.D., demographic information about the entity, ratings of the entity provided by other entities, a ratee reputation of the entity, a ratee reputation deviation of the entity, weighting values corresponding to attributes or rating parameters or items, goods, services, people, etc. associated with the entity, and other information regarding the entity.
- the authentication I.D. database 236 may include one or more entries, where each entry represents an entity that is part of a population using a ratee reputation system, a rater reputation system, or a combination thereof.
- An entry in the authentication I.D. database 236 may include an I.D. of an entity, demographic information about an entity, a password for the entity, and other information regarding an entity.
- the rater reputation database 238 may include one or more entries, where each entry represents data specific to an entity.
- an entry in the rater reputation database 238 may include an entity I.D., demographic information about the entity, ratings provided by the entity, a rater reputation of the entity, a rater reputation deviation of the entity, and other information regarding the entity.
- Each of the databases 234 , 236 and 238 may be of any type of a variety of types of databases, including a relational database (e.g., Microsoft SQL or Oracle) that stores data as entries in tables, an object-oriented database that stores data as objects or a flat file database, where entries are stored as records and separated by delimiters. Further, each of the databases 234 , 236 and 238 may each be part of single database. For example, a single database may maintain an entry for each entity, where each entry includes an I.D. of the entity, authentication data of the entity, ratings provided by the entity, ratings of the entity provided by other entities, a ratee reputation of the entity, weighting values, and a rater reputation of the entity, as well as other information.
- a relational database e.g., Microsoft SQL or Oracle
- object-oriented database that stores data as objects or a flat file database
- each of the databases 234 , 236 and 238 may each be part of single database.
- a single database may maintain an entry for each entity,
- the server 212 may include a frontend 230 and a backend 232 .
- the frontend 230 may include a user interface and may receive and send instructions and other data to and from the client 210 .
- the frontend 230 may then transfer and receive instructions to and from the backend 232 .
- the backend may determine ratee reputations, rater reputations, ratee reputation deviations, rater reputation deviations, weightings, and other metrics and values such as, for example, as described above in relation to FIGS. 2-17 .
- the backend 232 may also receive queries 216 submitted to the frontend 230 and translate these queries to database queries on one of the databases 234 , 236 or 238 .
- the backend 232 may also transfer updated reputations 244 , ratings 246 and weightings to the various databases.
- the client 210 and the server 212 may be implemented as software, firmware, or hardware, or any combination thereof.
- the client 210 and the server 230 may be implemented as software using any of a variety of programming languages, such as Java, C and C++.
- an entity may submit one or more ratings 224 to the server 212 . Included along with the ratings 224 may be a rater ID that identifies the rating entity and a ratee ID that identifies the entity being rated.
- the server 212 may then determine a ratee reputation, a rater reputation, weightings, and other information in accordance with the various techniques described above in relation to FIGS. 2-17 .
- the server 212 may submit a ratee query 240 to the ratee reputation database 234 and extract various data corresponding to the rated entity, including the current ratee reputation and previous ratings of the rated entity provided by other entities. This data may be sent from the ratee reputation database 234 to the server 212 as query results 242 . The server 212 then may use the query results 242 and the received ratings 224 to determine a ratee reputation of the rated entity according to any of a variety of the techniques described above , then may send the ratee reputation 226 through the frontend 230 to the client 210 .
- the server 212 may store or persist the one or more ratings 224 and the updated ratee reputation 244 by sending to the ratee reputation database 234 , for example, to an entry corresponding to the rated entity, and by sending the one or more ratings 224 and the updated ratee reputation to the rater reputation database 238 , for example, to an entry corresponding to the rating entity.
- the server 212 may submit a rater query 248 to the rater reputation database 238 and extract various data corresponding to the rating entity, including the current rater reputation and previous ratings of other entities provided by the rating entity. This data may be sent from the rater reputation database 238 to the server 212 as query results 250 . The server 212 then may use the query results 250 and the received ratings 224 to determine a rater reputation of the rated entity according to any of the techniques described above through the frontend 230 , the server 212 , then may send the determined rater reputation 226 to the client 210 .
- the server 212 may store the one or more ratings 224 and the updated rater reputation 244 by sending to the ratee reputation database 234 , for example, to an entry corresponding to the rated entity, and by sending the one or more ratings 224 and the updated rater reputation 244 to the rater reputation database 238 , for example, to an entry corresponding to the rating entity.
- the server 212 may use similar techniques to access the ratee reputation database 234 and the rater reputation database 238 to determine other values as described above in relation to FIGS. 2-17 , including determining ratee reputation deviations, rater reputation deviations, personalized ratee reputations, weightings, estimated results, and estimated result deviations.
- the server 212 may access the necessary values in the ratee reputation database 234 and/or the ratee reputation database 238 , determine resulting weighting values, for example, as described above in relation to FIGS. 12-15 , and then sending the weighting values 214 to the client 210 .
- the server 212 may receive a user query 216 from the client 210 .
- the user query may be requesting the rater reputations of one or more entities, the ratee reputations of one or more entities, or the personalized ratee reputation of one or more entities from the perspective of a particular entity.
- the server 212 may convert the user query 216 into a database query, for example, a ratee query 240 or a rater query 248 , and send the database query to the appropriate database.
- the server 212 then may send the query results 218 to the client 210 .
- the server 212 may be part of an on-line marketplace, for example, an agent-mediated marketplace. Accordingly, the client may request and receive marketplace information 220 from the server 230 . Further, the server 212 , as part of a transaction between an entity corresponding to the client 210 and a counterpart entity, may send communications 222 to the client 210 .
- the communications 222 may include notifications pertaining to the current transaction, prompts for information from the entity corresponding to the client 210 , reputations of the counterpart entity, and other information about the entity including demographic data, weighting values, etc.
- the client 210 may send instructions 228 to make a deal, for example, instructions to purchase a good.
- the client 210 may include a user interface to allow interaction between a user and an application, for example, a reputation application or marketplace application implemented using the client 210 and server 212 .
- the user interface may involve using CGI scripts to generate web pages in accordance with any of a variety of markup languages such as, for example, HTML, XML or SGML.
- the several methods described herein, and the various embodiments thereof, including the methods and embodiments described in relation to FIGS. 2, 3 , 8 , 10 , 12 , 14 , 16 and Equations 5-35, may be stored on a computer-readable medium as computer-readable signals.
- the computer-readable signals may define instructions that, in response to being executed on a computer, perform the method or embodiment.
- Each method or embodiment thereof may be implemented using software, firmware, hardware, or any combination thereof.
- the computer-readable signals defining the method or embodiment may be part of a computer program product, such as a software program written in any of a number of languages such as, for example, C, C++ or Java.
Abstract
A method and system for ascribing a reputation of an entity as a rater of other entities is provided. A first rating indicative of a rating of a rated entity by the first entity, and one or more second ratings, each second rating indicative of a rating of the rated entity provided by another entity, are provided. The second ratings are combined to produce a first combined rating. The first rating is compared to the first combined rating to produce a first rating predictability of the first rating, the first rating predictability being a negative function of a magnitude of a difference between the first rating and the first combined rating. A resulting rater reputation is generated based at least in part on the first rating predictability.
Description
- This application claims the benefit under 35 U.S.C. §120 of U.S. application Ser. No. 09/710,008, titled “Method And System For Ascribing A Reputation To An Entity As A Rater Of Other Entities,” filed on Nov. 10, 2000, which claims the benefit under 35 U.S.C. § 119(e) of commonly-owned, U.S. provisional patent application Ser. No. 60/209,059, titled, “Determining a Reputation of an Entity”, filed Jun. 2, 2000, both of which are incorporated herein by reference in their entirety.
- Further, each of the following related commonly-owned U.S. patent applications: “Method and System for Ascribing a Reputation to an Entity from the Perspective of Another Entity” by Giorgos Zacharia, “System and Method for Estimating the Impacts of Multiple Ratings on a Result” by Giorgos Zacharia, “System and Method for Ascribing a Reputation to an Entity” by Giorgos Zacharia, and “System and Method for Recursively Estimating a Reputation of an Entity” by Giorgos Zacharia, each application filed on Nov. 10, 2000, is herein incorporated by reference in its entirety.
- The emergence of the Internet and other large networks has increased both the number and kinds of electronic exchanges between entities. As used herein, an electronic exchange is any exchange between two or more entities over an electronic network (i.e., not in person) such as, for example, a voice communications network (e.g., POTS or PBX) or a data communications network (e.g., LAN or the Internet) or a voice and data communications network (e.g., voice-over-IP network). Electronic exchanges may include electronic business transactions and electronic communications. Such electronic business transactions may include the negotiation and closing of a sale of goods or services, including solicitation of customers, making an offer and accepting an offer. For example, in consumer-to-consumer electronic marketplaces (e.g., the eBay, OnSale, Yahoo and Amazon marketplaces found on the global Internet entities may transact for the sale and purchase of goods or services.
- Electronic communications also may include communications in on-line communities such as mailing lists, news groups, or web-based message boards and chatrooms, where a variety of sensitive personal information may be exchanged, including health-related data, financial investment data, help and advise on research and technology-related issues, or even information about political issues. As referred to herein, an entity may be a person or an electronic agent (e.g., a software agent). Such a person may Act as an individual (i.e., on the person's own behalf) or as a representative (e.g., officer or agent) of a corporation, partnership, agency, organization, or other group. An electronic agent may act as an agent of an individual, corporation, partnership, agency, organization, or other group.
- In many electronic exchanges, an entity's identity may be anonymous to another entity. This anonymity raises several issues regarding trust and deception in connection to these exchanges. For example, an anonymous entity selling goods on-line may misrepresent the condition or worth of a good to a buyer without suffering a loss of reputation, business or other adverse effect, due to the entity's anonymity.
- One solution to the problems regarding trust and deception is to provide a reputation mechanism to determine and maintain a reputation or reliability rating of an entity. Typically, a reputation mechanism is intended to provide an indication of how reliable an entity is, i.e., how truly its actions correspond to its representations, based on feedback by other entities that have conducted an electronic exchange with the entity. Such feedback typically is provided by another entity in the form of ratings in a numerical (e.g., 1-5) or Boolean (e.g., good or bad) form. In some reputation mechanisms, an average of the ratings provided by other entities are calculated to produce the reputation rating of the entity. Consequently, such reputation mechanisms typically represent the reputation of an entity with a scalar value.
- Typical reputation mechanisms suffer from susceptibility to frauds or deceptions. For example, a first typical fraud occurs when an anonymous entity, after developing a poor reputation over time in an on-line community, reenters the community with a new anonymous identity (i.e., on-line name), thereby starting anew with a higher reputation than the entity's already earned poor reputation. A second typical fraud, to which typical reputation mechanisms are susceptible, occurs when two or more entities collude to provide high ratings for each other on a relatively frequent basis, such that the reputations of these entities are thereby artificially inflated.
- Two reputation mechanisms that solve these two problems, Sporas and Histos, are disclosed in “Collaborative Reputation Mechanisms for On-line Communities” by Giorgos Zacharia, submitted to the Program of Media Arts and Sciences, Massachusetts Institute of Technology, Cambridge, Mass. published September, 1999 (hereinafter “the Zacharia thesis”), the contents of which are herein incorporated by reference.
- Sporas is a reputation mechanism for loosely-connected communities (i.e., one in which many entities may not have had an electronic exchange with one another and thus not have rated one another.) According to the Sporas technique, a reputation may be calculated for an entity by applying the following equation:
where Ri−1 is the initial reputation of the entity, C is an effective number of ratings, 1/C is the change rate factor, named as such because it impacts the rate at which the reputation changes, damp (Ri−1) is a damping function, Ri otheris the reputation of another entity providing the rating, Wi is the rating of the entity provided by the other entity, Ei is the expected value of the rating and Ri is the reputation of the entity. - Zacharia discloses that the damping function may be calculated by applying the following equation:
where D is the size of the range of allowed reputation values and a is a so-called “acceleration” factor. The acceleration factor is named as such because its value controls a rate at which an entity's reputation changes. The Zacharia thesis further discloses that an expected rating, Ei can be calculated from the following equation: - (Throughout this application, if a value represented by a symbol from a current equation was described in connection with a previously-described equation, the description of the value will not be repeated for the current equation.)
- The Sporas technique implements an entity reputation mechanism based on the following principles. First, new entities start with a minimum reputation value, and build-up their reputations as a result of their activities on the system. For example, if a reputation mechanism has a rating range from 1 to 100, then an entity may start with an initial reputation value, R0, of 1. By starting with the minimum reputation value, Sporas reduces the incentive to, and effectively eliminates, that ability of an entity with a low reputation to improve the entity's reputation by reentering the system as a new anonymous identity.
- Second, the reputation of an entity never falls below the reputation of a new entity. This may be ensured by applying
equation 1 above. This second principle also reduces the incentive, and effectively prevents, an entity with a low reputation from reentering the system as a new anonymous entity. - Third, after each electronic exchange, the reputations of each of the two or more entities involved are updated according to the feedback or ratings provided by the other entities, where the feedback or ratings represent the demonstrated trustworthiness of the two or more entities in the latest exchange. For example, referring to
Equation 1 above, the ratee reputation Ri of an entity is updated for each new rating, Wi. - Fourth, two entities may rate each other only once. If two entities exchange more than once, then, for each entity, the reputation mechanism only applies the most recently submitted rating to determine the reputation of the rated entity. This fourth principle prevents two or more entities from fraudulently inflating their reputations, as describe above, by frequently rating each other with artificially high ratings.
- Fifth, entities with very high reputation values experience smaller rating changes after each update. This fifth principle is implemented by the damping function, damp(Ri−1), of
Equations - Sixth, the reputation mechanism adapts to changes in an entity's behavior. For example, a reputation may be discounted over time so that the most recent ratings of an entity have more weight in determining the ratee reputation of the entity. For example, in
Equation 1, above, ratings are discounted over time by limiting the effective number of ratings considered, C. - The Sporas reputation mechanism also weights the reputation of a rated entity according to the reputation, Rother, of another entity providing the rating, where this reputation of the other entity may be determined by applying
Equation 1. Therefore, ratings from entities having relatively higher reputations have more of an impact on the reputation of the rated entity than ratings from entities having relatively lower reputations. - As described in the Zacharia thesis, Histos is a reputation mechanism better-suited for a highly-connected community, where entities have provided ratings for a significant number of the other entities. Histos determines a personalized reputation of a first entity from a perspective of a particular entity.
- Histos represents the principle that a person or entity is more likely to trust the opinion of another person or entity with whom she is familiar than trust the opinion of another person or entity who she does not know. Unlike Sporas, a reputation of first entity in Histos depends on the second entity from whose perspective the determination is made, and other ratings of the second entity provided by other users in an on-line community or population.
-
FIG. 1 is a block diagram illustrating a representation of an on-line community orpopulation 300 of entities A1-A11 interconnected by several rating links, includingrating links rating link 302 represents a rating of 0.8 for ratee A3 by rater A1, andrating link 303 represents a rating of 0.9 for ratee A1by rater A3. - Although in
FIG. 1 , each rating link only indicates a single rating, it is possible that an entity has provided more than one rating for another entity. The Zacharia reference discloses that if entity has provided more than rating for another entity, the most recent rating should be selected to determine a personalized reputation of a first entity from the perspective of a second entity. - To determine a personalized reputation of a first entity from the perspective of a second entity, the first and second entity must be “connected”. A first and second entity are connected if a rating path connects the first and second entity. A rating path is a series of rating links that connect a first entity to a second entity. For example, in
FIG. 1 , entities A1 and A11 are connected by several rating paths, includingrating paths Rating path 312 includesrating links rating path 314 includesrating links - As described in the Zacharia thesis, and referring to
FIG. 1 , to determine a personalized reputation of a first user from the perspective of a second user, the following methodology may be applied. First, a breadth-first search algorithm is applied to find all of the rating paths connecting A1 to A11 that are of a length less than or equal to a specified value. If a rating link indicates more than one rating, then the most recent rating is selected for the determination of the personalized reputation. - The number of rating links included in a rating path is referred to herein as the “length” of the rating path. For example, the
rating path 312 has a length=3 because it includes threerating links FIG. 1 , the entity A8 is disposed along therating path 314. The entity A8 has alevel 2 in the context of therating path 314 because tworating links - Accordingly, the personalized reputation of a first entity from the perspective of a second entity may be determined by application of the following equation:
where Rk(n) is the personalized ratee reputation of an entity k from a perspective of a second entity a distance n from the entity k, Wjk(n) is a rating provided by an entity j for the entity k a distance n−1 from the second entity, Rj(n−1) is the personalized ratee reputation of the entity j from the perspective of the second entity, and D is a range of allowable personalized reputation values. - Referring to
FIG. 1 , the following example illustrates applyingEquation 4 to determine the personalized ratee reputation of entity A11 from the perspective of entity A1, where D=1. - The Sporas reputation mechanism described in the Zacharia thesis uses a single reputation value to represent the reputation of an entity both as a ratee and a rater. This single reputation represents the reputation of an entity as a combination of ratings provided by other entities, but does not provide an indication of the reputation or trustworthiness of an entity as a rater of other entities. Thus, an entity may receive high ratings from other entities and thus have a high single-valued reputation, although the entity is a poor rater of other entities and thus should not truly have a high reputation as a rater of other entities. As used herein, a “rater reputation” of an entity is a reputation or trustworthiness of the entity as a rater of one or more other entities, and a “ratee reputation” of an entity is the reputation of the entity according to the ratings of the entity provided by one or more other entities.
- Therefore, Sporas does not provide a method or system for determining and maintaining a rater reputation of an entity.
- Accordingly, provided herein is a method and system for determining a rater reputation of an entity. Further, provided is a method and system for determining a ratee reputation of an entity based at least in part on the rater reputations of one or more of the entities that have rated the rated entity.
- In one embodiment, a rater reputation of a first entity as a rater of other entities is determined. A first rating indicative of a rating of a rated entity by the first entity, and one or more second ratings, each second rating indicative of a rating of the rated entity provided by another entity, are provided. The second ratings are combined to produce a first combined rating. The first rating is compared to the first combined rating to produce a first rating predictability of the first rating, the first rating predictability being a negative function of a magnitude of a difference between the first rating and the first combined rating. A resulting rater reputation is generated based at least in part on the first rating predictability.
- This embodiment may be implemented as a computer program product that includes a computer readable medium and computer readable signals stored on the computer readable medium that define instructions. These instructions, as a result of being executed by a computer, instruct the computer to perform the Acts described above for this embodiment.
- In another embodiment, a system for determining a rater reputation of a first entity as a rater of other entities is provided. A first rating indicative of a rating of a rated entity by the first entity, and one or more second ratings, each second rating indicative of a rating of the rated entity provided by another entity, is provided to the system. The system includes a rater reputation generator to combine the second ratings to produce a first combined rating. The rater reputation generator is further operative to compare the first rating to the first combined rating to produce a first rating predictability of the first rating, the first rating predictability being a negative function of a magnitude of a difference between the first rating and the first combined rating. The rater reputation generator is further operative to generate a resulting rater reputation based at least in part on the first rating predictability, and to output this generated rater reputation.
- In yet another embodiment, a system for determining a rater reputation of a first entity as a rater of other entities is provided. A first rating indicative of a rating of a rated entity by the first entity, and one or more second ratings, each second rating indicative of a rating of the rated entity provided by another entity are provided for the system. The system includes: means for combining the second ratings to produce a first combined rating; means for comparing the first rating to the first combined rating to produce a first rating predictability of the first rating, the first rating predictability being a negative function of a magnitude of a difference between the first rating and the first combined rating; and means for generating a resulting rater reputation based at least in part on the first rating predictability.
- In another embodiment, a ratee reputation of a first entity is determined. A first rating of the first entity by a second entity is received. One or more rater reputations including a first rater reputation of the second entity as a rater of other entities are accessed, and a ratee reputation of the first entity is generated by combining the one or more rater reputations and the first rating.
- This embodiment may be implemented as a computer program product that includes a computer readable medium and computer readable signals stored on the computer readable medium that define instructions. These instructions, as a result of being executed by a computer, instruct the computer to perform the Acts described above for this embodiment.
- In another embodiment, a system for determining a ratee reputation of a first entity is provided. The system includes a ratee reputation generator to receive as input a first rating of the first entity by a second entity. The ratee reputation generator is operative to access one or more rater reputations including a first rater reputation of the second entity as a rater of other entities, and to generate a ratee reputation of the first entity by combining the one or more rater reputations and the first rating signal. The ratee reputation generator is further operative to provide as output the generated ratee reputation.
- In yet another embodiment, a system for determining a ratee reputation of a first entity is provided. The system includes: means for receiving a first rating of the first entity by a second entity; means for accessing one or more rater reputations including a first rater reputation of the second entity as a rater of other entities; and means for generating a ratee reputation of the first entity by combining the one or more rater reputations and the first rating signal.
- The features and advantages of the invention described above and other features and advantages of the invention will be more readily understood and appreciated from the detailed description below, which should be read together with the accompanying drawing figures.
- In the drawings:
-
FIG. 1 is a block diagram illustrating a representation of an on-line community of entities interconnected by several rating links; -
FIG. 2 is a flowchart illustrating an example embodiment of a method of determining a rater reputation of an entity; -
FIG. 3 is a flow chart illustrating an example embodiment of a method of combining rating predictabilities to produce a rater reputation; -
FIG. 4 is data flow diagram illustrating an example embodiment of a system for generating a rater reputation of an entity; -
FIG. 5 is a data flow diagram illustrating an example embodiment of a system for generating a rater reputation of an entity; -
FIG. 6 is a data flow diagram illustrating an example embodiment of a system for generating a ratee reputation deviation of an entity; -
FIG. 7 is a data flow diagram illustrating an example embodiment of a system for generating a rater reputation deviation of an entity; -
FIG. 8 is a flowchart illustrating an example embodiment of a method of generating a ratee reputation of an entity; -
FIG. 9 is a data flow diagram illustrating an example embodiment of a system for generating a ratee reputation of an entity; -
FIG. 10 is a flowchart illustrating an example embodiment of a method of determining a personalized ratee reputation; -
FIG. 11 is a data flow diagram illustrating an example embodiment of a system for determining a personalized ratee reputation; -
FIG. 12 is a flow chart illustrating an example embodiment of recursively estimating the impact of one or more ratings on a result; -
FIG. 13 is a data flow diagram illustrating an example embodiment of a system for recursively estimating the impact of one or more ratings on a result; -
FIG. 14 is a flowchart illustrating an example embodiment of a method of generating a weighting vector for a set of multiple ratings; -
FIG. 15 is a data flow diagram illustrating an example embodiment of a system for generating a weighting vector for a set of multiple ratings; -
FIG. 16 is a flow chart illustrating an example embodiment of a method of combining multiple estimated reputations to produce a new reputation; -
FIG. 17 is a data flow diagram illustrating an example embodiment of a system for combining multiple estimated reputations to produce a new estimated reputation; and -
FIG. 18 is a data flow diagram illustrating an example system architecture for implementing the methods and systems ofFIGS. 2-17 . - I. Determining a Rater Reputation
- Described below is a method and system for determining a rater reputation of a first entity. Although determining a rater reputation is described below primarily in connection to ratings developed in connection with electronic exchanges, such determination may be applied to any of a variety of ratings, regardless of whether the rating is provided as a result of an electronic exchange. For example, a rating may represent a qualitative assessment of an in-person interview of a job candidate, an in-person sale or a credit transaction.
- The rater reputation produced using the method and system described below may have any of a variety of applications such as, for example, assessing the trustworthiness (as a rater) of a reference, an employment recruiter, a rating agency, or any other entity that provides recommendations, scores, rankings or ratings of other entities.
- A rater reputation may be determined, for one or more ratings of one or more rated entities, respectively, provided by a first entity, by comparing each rating to other ratings of the rated entity provided by other entities. For each rated entity, the statistical similarity between the rating provided by the first entity and ratings provided by the other entities as determined by the comparison, i.e., the rating predictability, may serve as a basis for a rater reputation of the first entity.
- Optionally, the ratings provided by the other entities may be ratings provided at points in time that occur after a point in time at which the rating by the first entity is provided. Determining a rater reputation of a first entity by comparing ratings provided by the first entity to such future ratings may provide a more accurate estimate of the first entity's rater reputation than comparing these ratings to past ratings of other entities. This more accurate estimate may result from the fact that if past ratings are used for the comparisons, and the first entity is aware that past ratings are being used, the first entity may access the past ratings and bias her ratings to be consistent with the past ratings, thus resulting in a higher reputation of the first entity than otherwise would occur.
- Accordingly, by using future ratings, which are necessarily unknown to the first user, for the comparisons, the first entity has less incentive to bias her ratings, resulting in more honest ratings being provided by the first entity. These more honest ratings result in a more accurate estimation of the rater reputation of the first entity.
- In an aspect of determining a rater reputation of an entity, the result of the comparison of (a) the rating provided by the first rater of a rated entity and (b) other ratings of the rated entity provided by other raters may be weighed over a ratee reputation deviation of the rated entity, as will be described in more detail below in relation to
FIG. 5 . This ratee reputation deviation represents a deviation of ratings of the rated entity from an expected value of the rating of the rated entity. Entities whose ratee reputations fluctuate over a wide range of values, such as new entities and entities that receive a wide range of ratings (i.e., unstable entities), typically have high ratee reputation deviations. This weighting of the comparison results in a rating predictability that is greater for greater values of ratee reputation deviation and less for lesser values of ratee reputation deviation. -
FIG. 2 is a flowchart illustrating an example embodiment of a method of generating a rater reputation of an entity, where a first rating of a rated entity by a first entity and second ratings of the rated entity provided by other entities are provided. The first and second ratings may be provided by accessing two or more entries of a data structure such as, for example, a database, stored on a computer-readable medium such as, for example, a non-volatile memory (e.g., a magnetic disk, CD ROM, or magnetic tape) or a volatile memory (e.g., an integrated semiconductor memory such as RAM). Such a data structure may reside on a computer-readable medium located on a same computer at which an application running the method of generating resides and may be accessed by a bus or other known means. Alternatively, the data structure may reside at a remote location from such application, where access of the data structure may include use of one or more networks, bridges, routers, switches, hubs, other network devices, or any combination thereof. Such access may include wireless and wire (i.e., cable) transmissions. - In
Act 2, two or more of the second ratings of the rated entity may be combined to produce an expected rating. This expected rating represents an expected value of the first rating in consideration of the values of the two or more second ratings. The two or more ratings to be combined may be selected according to each rating's temporal proximity to the rating provided by the first entity (i.e., the first rating), or may be selected with other criteria such as, for example, demographic similarity of the entity that provided the second rating to the first entity. Optionally, only second ratings provided temporally after the first rating are selected. Such selected second ratings may be referred to herein as future ratings. Although all of the second ratings may be selected, which may produce a more accurate expected rating, the computational time and cost will increase as the number of second ratings combined increases; consequently, it may be desirable to select less than all of the second ratings. - A limiting factor may be provided to determine a number of the second ratings to select. The value of the limiting factor may be selected to achieve an acceptable balance between (a) rater reputation accuracy and (b) the time and cost of determining the rater reputation.
- In an aspect of selecting second ratings, second ratings provided by the first entity are excluded from selection. Such exclusion prevents the first entity from providing multiple ratings of the rated entity, and intentionally providing a same or similar value as each of the multiple ratings to artificially increase the first entity's rater reputation.
- The selected second ratings may be combined by calculating an average of the second ratings. Optionally, this calculated average may be weighted, where each selected second rating is weighted according to the reputation of the entity that provided the second rating relative to the reputation of the other entities that provided second ratings. For example, a weighted average of second ratings provided by other entities may be determined by applying the following equation:
where k-j is the number of selected second ratings, Ejk ratee,other is the weighted average of the k-j second ratings, Wu is the rating provided by a second entity u, and Ru rater is the rater reputation of second entity u. Although applyingEquation 5 represents combining only future rating (from j+1 to k), other second ratings may be combined. For example, inEquation 5, above, u may begin at u=j−1−C/2 and end at k=j−1+C/2, where 1/C is a change rate factor, described in more detail below in relation toAct 12. A situation in which it may be desirable to determine an expected rating by combining selected second ratings other than future ratings is described below in more detail in relation to “seeding” a system. - Weighting each second rating according to the reputation of the entity that provided the second rating generates an expected rating that gives more weight to ratings by those entities that have higher rater reputations. For example, if most of the higher second ratings were provided by entities having a low rater reputation, and most of the lower second ratings were provided by entities having a higher rater reputation, then the calculated expected rating would be lower than a raw average. This lower expected rating represents a bias towards the second ratings provided by the entities having a higher rater reputation. Such weighting of each selected second rating represents the principle that entities having higher rater reputations give more accurate ratings, and thus the ratings they provide are more valuable in determining a rater reputation of an entity (by definition) and, therefore, should be given more weight than ratings provided by entities having lower rater reputations.
- The average rating of the selected second ratings also may be weighted according to the relative proximity of the times at which the selected second ratings are provided to the time at which the first rating is provided. In other words, a rating provided closer in time to the first rating time will have a greater weight and thus more of an impact on the calculated average than a temporally more distant rating. This weighting reflects the principle that an entity's ratee reputation may change over time. Thus, for a given time at which a first rating is compared to other ratings, temporally closer ratings should be given more weight in determining the rater reputation of the first entity. Conversely, temporally more distant ratings should have lesser weight in determining the rater reputation of the first entity.
- If only second ratings occurring after the first point in time (i.e., future ratings) are selected, then the earlier the second rating is provided, the greater is the weight attributed to the second rating. Further, time may be divided into a number of intervals, where each second ratings is provided during one of the intervals. Each second rating may be weighted according to the temporal proximity to the first point in time of the interval in which the second rating was provided.
- Optionally, each selected second rating may be placed in a temporal order according to its relative temporal proximity to the first point in time, and each selected second rating may be weighted according to its position in this order. If only future ratings are selected, then the earlier the second rating is provided, the lower (i.e., the closer to the beginning) the rating is in the temporal order. Thus, a temporally-weighted average may be defined by the following equation:
where ƒ(u) is a temporal function such as, for example:
where T is a constant number having a value>1, and u may be: the position of rating Wu in a temporal order; a temporal proximity of the rating from the first point in time; or a temporal proximity of a time interval in which the rating was provided to the first point in time. - Combining
Equations - Other statistical methods may be used to determine an expected rating of an entity.
- Returning to the method of
FIG. 2 , next, inAct 4, the first rating of the rated entity, provided by the first entity, may be compared to the expected rating, to produce a rating predictability. This rating predictability may be a negative function of an absolute difference between the first rating and the expected rating. As used herein, a first value is a negative function of a second value if the first value decreases as a result of the second value increasing, and increases as a result of the second value decreasing. Further, as used herein, a first value is a positive function of a second value if the first value increases as a result of the second value increasing, and decreases as a result of the second value decreasing. - For example, comparing the first ratings and the second ratings may include determining the absolute difference (i.e., magnitude of the difference) between the first rating and the second rating and then applying a Gaussian distribution function to the determined absolute difference to produce the first rating predictability. If a Gaussian distribution function is being applied, then the absolute difference between the first rating and the expected rating may first be divided by a ratee reputation deviation of the rated entity. The ratee reputation deviation defines a deviation of ratings by other entities of the rated entity from an expected rating, and is described in more detail below in relation to
FIG. 6 . Thus, a rating predictability for the first rating may be determined by applying the following equation:
where
is the ratee reputation deviation of the rated entity, and Pj(X) is the rating predictability of the first rating Wj. - Dividing the absolute difference by the rates reputation duration has the effect of weighting the absolute difference over the ratee reputation deviation. This weighting may be desirable to account for the stability or reliability of rater ratings of the rated entity in determining the rates reputation of the first entity. For example, if the ratee reputation of the rated entity is unstable (i.e., ratings provided for the rated entity vary considerably), the rater reputation of the first entity should be changed less by the difference between the expected rating and the first rating. Conversely, if the ratee reputation of the rated entity is stable, the rater reputation of the first entity should be changed more by the difference.
- Alternatively, if the first entity has provided other ratings of other entities, and rating predictabilities have been generated from these other ratings (e.g., by performing
Acts Act 5, the other rating predictabilities may be combined with the rating predictability generated inAct 4 to produce the rater reputation of the first entity.Act 5 may be implemented in any of several different ways. In one implementation,Act 5 may be implemented by averaging all of the generated rating predictabilities associated with each rating provided by the first entity. Accordingly, a rater reputation may be determined by applying the following equation:
where
is the minimum allowed ratee reputation deviation. Other methods may be used to determine a rating predictability of an entity. - If the first rating is a first-time rating by the first entity, then the generated rating predictability may alone serve as the rater reputation of the first entity.
- Alternatively, if the first entity has provided other rating of other entities, and rating predictabilities have been generated from these other ratings (e.g., by performing
Acts Act 5, the other rating predictabilities may be combined with the rating predictability generated inAct 4 to produce the rater reputation of the first entity.Act 5 may be implemented in any of several different ways. In one implementation,Act 5 may be implemented by averaging all of the generated rating predictabilities associated with each rating provided by the first entity. Accordingly, a rater reputation may be determined by applying the following equation:
where n is the number of ratings provided by the first entity and Rrater is the rater reputation. - If the rater reputation of the first entity is determined by averaging the first entity's rating predictabilities, the average may be a weighted average. For example, each rating predictability may be weighted as a positive function of the time at which the rating associated with the rating predictability is provided. For example, the earlier the associated rating is provided, and thus the further away in time from when the rater reputation is being determined, the lower the weighting of the predictability. Optionally, each rating predictability may be placed in a temporal order according to the time at which its associated rating was provided, and each predictability may be weighted according its position in this order. Further, time may be divided into a number of intervals, and the temporal intervals may be placed in a temporal order. A rating predictability then may be weighted according to the position of the temporal interval in the temporal order.
- Weighting the rating as such represents the principle that an entity's rater reputation may change over time, and thus more recent ratings should be given more weight than older ratings in determining the rater reputation of an entity. A rater reputation, determined by applying such a weighted average of predictabilities, may be determined by applying the following equation:
where ƒ(j) is a temporal function such as, for example:
where V is a constant number having a value<1, and j is the position of rating predictability, Pj, in a temporal order, or a time at which the rating associated with the predictability was provided, or a position in a temporal order of a temporal interval during which the associated rating was made. Other weightings and combinations of weightings may be applied to an average predictability to determine a rater reputation of an entity such as, for example, a combination of entity reputations associated with each predictability, described in detail below in relation toAct 10. - Determining a rater reputation of a first entity by averaging rating predictabilities of the first entity may become more cost prohibitive as the number of ratings of other entities provided by the first entity increases.
- Accordingly, as an alternative implementation of
Act 5, the rater reputation may be determined recursively. Determining a rater reputation recursively, for example, as described below, saves computational space and time, particularly as the number of ratings of other entities provided by the first entity grows. - Such a recursive determination may include: providing a previously determined rater reputation (i.e., an initial rater reputation) of the first entity; determining a rater reputation adjustment (which may be a positive or negative value) based on the generated rating, and adding the reputation adjustments to the previously determined rater reputation to produce the rater reputation.
-
FIG. 3 is a flow chart illustrating an example implementation ofAct 5. InAct 6, the rating predictability may be subtracted from the initial rater reputation to produce a reputation modification. Next, inAct 8, the reputations of the entities that provided the selected second ratings may be combined to produce a combined reputation such as, for example, by calculating an average reputation from these reputations. For example, a combined reputation may be determined by applying following equation:
where Rcombined is the combined reputation and Ru other is the rating of an entity that provided one of the selected second ratings.Act 8 may be performed in sequence or concurrently with Acts 2-6. - Next, in
Act 10, the reputation modification may be scaled by the combined reputation to produce a scaled reputation modification. This scaling has the effect of weighting the reputation modification according to the reputations of the entities corresponding to the second ratings. If the combined reputation is an average reputation of the entities that provided the selected second ratings and has a relatively high value, then the reputation modification is scaled such that the reputation adjustment is relatively high. Conversely, if the average rater reputation of these raters is relatively low, then the reputation adjustment resulting from the scaled reputation modification is relatively low. The scaled reputation modification may be determined by applying the following equation:
where
is the initial rater reputation,
is the reputation modification, and Rscaled is the scaled reputation modification. - Next, in
Act 12, the scaled reputation modification maybe divided by a change rate factor to produce a reputation adjustment. This change rate factor controls the rate at which the first rating and the resulting rating predictability can change the rater reputation of the first entity. For example, if the rate change factor has a high value, then the rate of change will be slower such that a relatively higher number of ratings will have to be provided by an entity before the rater reputation of the entity approaches an Actual reputation of the entity. - Next, in
Act 14, the reputation adjustment is added to the initial rater reputation to produce a resulting rater reputation. The resulting rater reputation may be determined by applying the following equation:
where 1/C is the change rate factor and Rj rater is the resulting rater reputation. The change rate factor is named as such because its value impacts the rate at which the rater reputation changes value. Optionally, the value of C may be set equal to the value of To, the effective number of observations, described below in relation toEquations - In an aspect of recursively determining a rater reputation of an entity, for example, by applying Equation 16, an original rater reputation (i.e., an entity's rater reputation before a first rater reputation has been determined for the entity), R0 rater, is initialized to zero. Consequently, an entity's rater reputation has its lowest value before the entity provides a first rating of another entity. Such an original rater reputation prevents an entity with a low rater reputation from creating a new identity as a new entity and beginning with a higher rater reputation than the low reputation of the entity.
- Recursively determining a rater reputation of a first entity, for example, by applying Equation 16, may result in an rater reputation that asymptotically approaches an Actual rater reputation of the first entity as the number or ratings of other entities provided by the first entity increases. An Actual rater reputation of the entity may be determined by averaging the determined rating predictabilities (and possibly weighting the average) for each rating provided by the first entity, for example, by applying
Equation - In an aspect of recursively determining a rater reputation of an entity, in response to receiving a rating of a rated entity, the rater reputations of the entities that provided the most recent M ratings may be determined. In other words, each of the selected second ratings is one of the most recent M ratings.
- Further, the determined rater reputation for each rating entity may be weighted as a function of how recent the rating entity provided its rating in relation to how recent the other rating entities provided their ratings. For example, for each of rating entities corresponding to one of the last M ratings received, the rating entity's rater reputation may be determined by application of the following equation:
where M is the number of most recent ratings, m is the number of ratings (of the most recent ratings) received after the rating received from the rating entity plus 1 (i.e., for the rating entity that provided the most recent rating, the value of m=1), and R(m)scaled is the scaled rater reputation modification as described above in relation to Equation 16, where, for the expected rating (as described in relation to Equation 8) from which the scaled rater reputation modification is generated, k−j=m. - For example, if rater reputations are being determined for the rating entities that provided the most recent 20 ratings, then, M=20. For the rating entity that provided the earliest rating, m=20 and m/M=1. Further, for the rating entity that provided the most recent rating, m=1 and m/M=1/20.
- By weighting the adjustment to the rater reputation of an entity by m/M, the amount that a rating entity's rater reputation is adjusted is proportional to the number of ratings received for a rated entity after the rating provided by the rating entity. This weighting reflects the principle that if there are more ratings received after the rating provided by the rating entity, then there is more information to estimate the predictability of the rating entity, as described above in relation to
Equations - Other techniques may be used for weighting a rater reputation adjustment as a function of how recent the rating entity provided its rating in relation to how recent the other rating entities provided their ratings.
- As described above, in an aspect of determining a rater reputation of an entity, each time a new entity enters a population or community of entities that use a rating system incorporating rater reputation, the new entity may have an original rater reputation initialized to zero. Initializing an original rater reputation to zero may be desirable if an entity is capable of having multiple identities. In a system where an entity is restricted to a lifetime persistent identity, for example, a social security number, then the original rater reputation may not be initialized to zero, but may be initialized to another value. For example, the original rater reputation may be initialized to an average rater reputation value.
- In some situations, an entire population of entities that have been using a rating system to rate each other, and thus have pre-collected ratings, may want to determine a rater reputation of one or more, possibly all, entities of the population using these pre-collected ratings. Such determination of rater reputations for a population of entities using pre-collected ratings may be referred to herein as “seeding” a reputation system. If seeding a reputation system, one or more first entities of the population may not have provided a sufficient number of ratings of other entities such that a recursively generated rater reputation of these first entities approximates an Actual rater reputation of these first entities, respectively. Consequently, the resulting rater reputation of one or more first entities that have provided relatively few ratings is lower than the resulting reputation of one or more first entities that have provided relatively many ratings, irrespective of the Actual ratings provided by the first entities.
- Accordingly, in an aspect of seeding a reputation system by recursively determining a rater reputation of a first new entity, pre-collected ratings may be used to determine the rater reputation as follows. In a first pass of determining the rater reputation, for example, by applying Equation 16 or variations thereof, the pre-collected ratings are used and the original rater reputation of the first new entity may be initialized to zero. A second pass of determining the rater reputation may use the same pre-collected ratings and initialize the original rater reputation of the first new user to the rater reputation resulting from the first pass. This initialization to a resulting reputation may be repeated in other passes until the resulting reputation adequately approximates the Actual rater reputation of the first new entity.
- Further, to seed a reputation system by recursively determining a rater reputation of a first entity, the expected rating for each first rating provided by the first entity may be determined by applying the following equation:
where Ri−1 is the ratee reputation of the rated entity at i-1, D is the range of allowed reputation values, 1/C is the change rate factor and Ei ratee is the determined expect rating. -
FIG. 4 is a data flow diagram illustrating an example embodiment of asystem 19 for generating arater reputation 38. Therater reputation generator 20 may receive arequest 21 from a user indicating a request for a first entity's reputation. In response to theuser request 21, therater reputation generator 20 may receive as input afirst rater rating 26 and selectedsecond ratings 28, and generate the resultingrater reputation 38 as output, for example, by performing Acts 2-5 ofFIG. 2 . In one implementation, therater reputation generator 20 may also receive as inputother rating predictabilities 49 to generate a resultingrater reputation 38 by averaging rating predictabilities, as described above in relation toFIG. 2 . - In another implementation, the
rater reputation generator 20 may also receive as input aninitial rater reputation 22,other rater reputations 24, aratee reputation deviation 30, arange 32, and achange rate factor 34, and use these inputs in addition toinputs rater reputation 38 such as, for example, by applying Acts 6-14 ofFIG. 3 . - In yet another implementation, the rater reputation generator further may receive a
value 31 indicating a number of most recent ratings and avalue 33 indicating the number of ratings provided for the rated entity after thefirst rater rating 26 was provided.Values -
FIG. 5 is a data flow diagram illustrating a more detailed example embodiment of thesystem 19 for generating a resultingrater reputation 38. Therater reputation generator 20 may include an expectedrating generator 40, apredictability generator 44, areputation modification generator 48, a combinedpredictability generator 47, a combinedreputation generator 52, areputation adjustment generator 56 and anadder 60. - In response to the
rater reputation generator 20 receiving theuser request 21, the expectedrating generator 40 may receive as input the selectedsecond ratings 28 and generate as output an expectedrating 42. The expectedrating generator 40 also may receive as inputother rater reputations 24, and use thisinput 24 in addition toinput 28, to generate the expectedrating 42. Theother rater reputations 24 may be the reputations of the entities that provided the selectedsecond ratings 28.Expected rating generator 40 may generate expectedratings 42 in accordance with the various techniques described above in relation toAct 2 ofFIG. 2 . - The
predictability generator 44 may receive as input the expectedratings 42 and thefirst rater rating 26 and produce arating predictability 46 as output. Thepredictability generator 44 also may receive theratee reputation deviation 30 and therange 32, and use both of these inputs to help produce therating predictability 46. Thepredictability generator 44 may generate therating predictability 46 in accordance with the various techniques discussed above in relation toAct 4 ofFIG. 2 . - The rater reputation of a first entity may be determined according to
Equations predictability generator 47 that receives asinput rating predictability 46 and other rating predictabilities 49 (which may have been generated by predictability generator 44) and generates as output a resultingrater reputation 38. - The
reputation modification generator 48 may receive arating predictability 46 and aninitial rater reputation 22, and generate areputation modification 50. Thereputation modification generator 48 may generate thereputation modification 50 by subtracting the rating predictability from the initial rater reputation as described above in relation toAct 6 ofFIG. 3 . - The combined
reputation generator 52 may receive otherrater reputations 24 and generate a combinedreputation value 54. The combinedreputation generator 52 may calculate the average of the other rater reputations to determine the combinedreputation 54. Areputation adjustment generator 56 may receive the combinedreputation 54 and thereputation modification 50 and generate areputation adjustment 58. Thereputation adjustment generator 56 may also receive thechange rate factor 34. Thereputation adjustment generator 56 may generate thereputation adjustment 58 in accordance with the techniques described above in relation toActs FIG. 3 . As described above, the combinedreputation 54 may scale thereputation modification 50 in accordance with the combinedreputation 54 of the other raters. - In an embodiment, the reputation adjustment generator further receives
values rater reputation adjustment 58 as described above in relation to Equation 17. - The
adder 60 may add theinitial rater reputation 22 to thereputation adjustment 58 to produce the resultingrater reputation 38. - The
rater reputation generator 20 and any components thereof, including expectedrating generator 40,predictability generator 44, combinedreputation generator 52, combinedpredictability generator 47,reputation modification generator 48,reputation adjustment generator 56 andadder 60, may be implemented as software, hardware, firmware or any combination thereof. Therater reputation generator 20 and any components thereof may reside on a single machine (e.g., a computer), or may be modular and reside on multiple interconnected (e.g., by a network) machines. Further, on each of the one or machines that include therater reputation generator 20 or a component thereof, thegenerator 20 or component may reside in one or more locations on the machine. For example, different portions of therater reputation generator 20 or different portions of a component may reside in different areas of memory (e.g., RAM, ROM, disk, etc.) on a computer. - The
range 32 and thechange rate factor 34 may be constants for thesystem 19 and may be stored in a reputation database or other data structure, as described in more detail below in relation toFIG. 18 . Thefirst rater rating 26, theother reputations 24, the selectedsecond ratings 28, theratee reputation deviation 30, theinitial rater reputation 22, and values 31 and 33 also may be stored in the reputation database or other structure. In response to receiving theuser request 21, therater reputation generator 20 may access the reputation database or similar data structure and retrievevalues rater reputation 38 as described above. The resulting raterreputation 38 then may be stored in the reputation database or other data structure for later access. -
System 19, and the components thereof, are merely example embodiments of a system for generating a rater reputation. Such example embodiments are not meant to limit the scope of the invention and are provided merely for illustrative purposes, as any of a variety of other systems and components for determining a rater reputation, determining a may fall within the scope of the invention. - II. Determining a Ratee Reputation Deviation
-
FIG. 6 is a data flow diagram illustrating an example embodiment of asystem 69 for recursively generating theratee reputation deviation 30 of a rated entity. Similarly to as described above in relation to recursively determining a rater reputation, determining a ratee reputation deviation recursively saves computational space and time, particularly as the number of provided ratings for the rated entity grows. - The ratee
reputation deviation generator 72 may receive as input aratee reputation 70, arater reputation 74, an initialratee reputation deviation 75, a mostrecent rating 76 and a forgettingfactor 77. The rateereputation deviation generator 72 may also receive therange 32. - The
ratee reputation 70 is the ratee reputation determined from the mostrecent rating 76 of the rated entity, for example, by applyingEquation 1 as disclosed in the Zacharia thesis, or by applying a variation ofEquation 1, where the rater reputation is determined in accordance with Equation 15 or a variation thereof, as described above in relation toFIG. 2 . The initialratee reputation deviation 75 is the ratee reputation deviation of the rated entity before the mostrecent rating 76 was provided. Therater reputation 74 is the rater reputation of an entity that provided the mostrecent rating 76. Theratee reputation generator 72 may generate from these inputs theratee reputation deviation 30 that represents a reliability of the rated entity's ratee reputation. - Optionally, the ratee
reputation deviation generator 72 also may receive a minimum allowed rater reputation deviation and compare it to the determinedrater reputation deviation 30. If the minimum deviation is greater thandeviation 30, then therater reputation deviation 30 may be set equal to the minimum allowed rater reputation. - Accordingly, the ratee
reputation deviation generator 72 may generate theratee reputation deviation 30 by applying the following equation:
where
is a minimum allowed ratee reputation deviation, F is the forgettingfactor 172,
is theinitial ratee reputation 75, Ri rater is therater reputation 74, Wi is the mostrecent rating 76, T0 is an effective number of reputation determinations, Ei rater,past is the expected rating, and RDi ratee is the resultingratee reputation deviation 30. The expected rating may be calculated by applyingEquation 2 as described in the Zacharia thesis. The forgetting factor is a constant having a value<1 such that older ratings have less weight in determining the resulting ratee reputation deviation than more recent ratings. T0 may be derived from F by applying the following equation: - Optionally, the effective number of determined reputations may be used to determine the change rate factor described above by setting C=T0.
- Other statistical methods may be used to determine a reliability of a ratee reputation. For example, a forgetting factor may not be applied, or an average deviation may be determined for all determined ratee reputations of the rated entity, or such an average deviation may be calculated with each ratee reputation weighted according to how recent the ratee reputation was determined.
- In an aspect of recursively determining a ratee reputation deviation of a rated entity, for example, by applying
Equation 19, an original ratee reputation deviation, RDi ratee, is initialized to a maximum allowed value for a ratee reputation deviation. Such a maximum value may be predetermined. Consequently, a rated entity's ratee reputation deviation has its highest value before the rated entity is rated by other entities. Assigning a maximum value to an original ratee reputation deviation prevents an entity with a high ratee reputation deviation from creating a new identity as a new entity and beginning with a more desirable lower ratee reputation deviation. -
Equation 19 may be considered a recursive estimation algorithm of Recursive Least Squares (RLS) with a forgetting factor ofF. Equation 19 estimates recursively an average square deviation of an actual rating from an expected (i.e., estimated) rating described in more detail below in relation toFIGS. 8 and 9 . For more information regarding Recursive Least Squares, please refer toChapter 9 of “Lecture Notes and Non-Linear and Non-Stationary Time Series Analysis,” by H. Madsen and J. Holst, Institute of Mathematical Modeling (IMM), Technical University of Denmark, Lyngby, Denmark, 1998 (hereinafter the Madsen text), the contents of which is herein incorporated by reference in its entirety. - Other statistical methods may be used to determine a reliability of a ratee reputation. For example, a forgetting factor may not be applied, or an average deviation may be determined for all determined ratee reputations of the rated entity, or such an average deviation may be calculated with each ratee reputation weighted according to how recent the ratee reputation was determined.
- The ratee
reputation deviation generator 72 may be implemented as software, hardware, firmware or any combination thereof. The rateereputation deviation generator 72 may reside on a single machine (e.g., a computer), or may be modular and reside on multiple interconnected (e.g., by a network) machines. Further, on each of the one or more machines that include the rateereputation deviation generator 72 or modules thereof, thegenerator 72 or modules may reside in one or more locations on the machine. For example, different portions of the rateereputation deviation generator 72 or modules may reside in different areas of memory (e.g., RAM, ROM, disk, etc.) on a computer. - The
range 32 and the forgettingfactor 77 may be constants stored in a reputation database or similar data structure as described in more detail in relation toFIG. 18 . Theratee reputation 70, therater reputation 74, and the initialratee reputation deviation 75 also may be stored in the reputation database or similar structure. The rateereputation deviation generator 72, in response to receiving the mostrecent rating 76, may access the reputation database or similar structure to generate theratee reputation deviation 30 as described above. Theratee reputation deviation 30 then may be stored in the reputation database or similar structure for later access. -
System 69 is merely an example embodiment of a system for generating a ratee reputation deviation. Such an example embodiment is not meant to limit the scope of the invention and is provided merely for illustrative purposes, as any of a variety of other systems for determining a ratee reputation deviation may fall within the scope of the invention. - III. Determining a Rater Reputation Deviation
- In addition to having an estimate of the reliability of the ratee reputation, for example, the
ratee reputation deviation 30, it may be desirable to have an estimate of the reliability of the rater reputation of an entity that is providing a rating. Such a reliability may be estimated by calculating of a rater reputation deviation. Similarly to as described above in relation to ratee reputation deviation, computational space and time may be saved by calculating a rater reputation deviation recursively, particularly as the number of ratings of other entities provided by the first entity increases. - Accordingly, a rater reputation deviation may be determined recursively by applying the following equation:
where
is a minimum allowed rater reputation deviation,
is an initial rater reputation deviation, RDj rater is the resulting rater reputation deviation, and the other symbols are as described above. -
Equation 21, and variations thereof, estimate recursively an average square deviation of a rater reputation of an entity from an expected (i.e., estimated) rater reputation. For each recursive estimate, the initial rater reputation deviation,
may be weighted by the forgetting factor, F, and the average square deviation may be divided by the effective number of determined reputations, T0. Such recursive estimation is described in more detail inchapter 9 of the Madsen text. - Other statistical methods may be used to determine a reliability of a rater reputation. For example, a forgetting factor may not be applied, or an average deviation may be determined for all determined rater reputations of the rating entity, or such an average deviation may be calculated with each rater reputation weighted according to how recent the rater reputation was determined.
- In an aspect of recursively determining a ratee reputation deviation of a first entity, for example, by application of
Equation 21, an original rater reputation deviation, RDj rater, is initialized to a maximum value. This maximum value may be predetermined. Consequently, a first entity's rater reputation deviation has its highest value before the first entity provides a first rating of another entity. Assigning a maximum value for the rater reputation deviation prevents a first entity with a high rater reputation deviation from creating a new identity as a new entity and beginning with a lower rater reputation deviation. -
FIG. 7 is a data flow diagram illustrating an example embodiment of asystem 79 for generating a rater reputation deviation. A raterreputation deviation generator 82 may receive as input aninitial rater reputation 22, arating predictability 46, an initialrater reputation deviation 80 and a forgettingfactor 77. Raterreputation deviation generator 82 also may receive otherrater reputations 24. - The rater
reputation deviation generator 82 may generate therater reputation deviation 84 frominputs inputs deviation generator 82 may receive the scaled reputation modification as input and generate therater reputation deviation 84 in accordance withEquation 21 or variations thereof. - Optionally, the rater
reputation deviation generator 82 may receive a minimum allowed rater reputation deviation,
and compare it to the determinedrater reputation deviation 84. If the minimum deviation is greater thandeviation 84, then therater reputation deviation 84 may be set equal to the minimum allowed rater reputation. - The rater
reputation deviation generator 82 may be implemented as software, hardware, firmware or any combination thereof. The raterreputation deviation generator 82 may reside on a single machine (e.g., a computer), or may be modular and reside on multiple interconnected (e.g., by a network) machines. Further, on each of the one or more machines that include the raterreputation deviation generator 82 or modules thereof, thegenerator 82 or modules thereof, thegenerator 82 or modules may reside in one or more locations on the machine. For example, different portions of the raterreputation deviation generator 82 may reside in different areas of memory (e.g., RAM, ROM, disk, etc.) on a computer. - The forgetting
factor 77 may be stored as a constant in a reputation database or similar data structure as described below in relation toFIG. 18 . Theinitial rater reputation 22, therating predictability 46, theother rater reputations 24 and the initialrater reputation deviation 80 also may be stored in the reputation database or similar data structure. In response to receiving therating predictability 46, the raterreputation deviation generator 82 may access the reputation database or similar data structure to access and retrievevalues rater reputation deviation 84. Therater reputation deviation 84 then may be stored in the reputation database or similar structure for later access. -
System 79 is merely an example embodiment of a system for generating a rater reputation deviation. Such an example embodiment is not meant to limit the scope of the invention and is provided merely for illustrative purposes, as any of a variety of other systems and components for determining a rater reputation deviation may fall within the scope of the invention. - IV. Determining a Ratee Reputation Using a Rater Reputation
- If a rater reputation is determined for a first entity, and the first entity provides a first rating of a rated entity, the rater reputation and the first rating may be used to determine, at least in part, a ratee reputation of the rated entity.
- Accordingly, provided herein is a method and system for determining a ratee reputation of a rated entity based at least in part on one or more ratings of the rated entity provided by one or more entities and the rater reputations of these one or more entities.
- In one embodiment of determining a ratee reputation, the ratee reputation of a rated entity is determined by averaging all of the ratings provided by other entities for the rated entity, and by weighting each rating with the rater reputation of the entity that provided the rating, to produce a weighted average. Accordingly, a ratee reputation may be determined by applying the following equation:
where Wj is a rating supplied, Rj rater is the rater reputation of the entity who provided the rating Wj. - In generating such a weighted average, each rating may be further weighted with a positive function of the time at which the rating was provided. For example, the earlier the rating is provided, and thus the further away in time from when the ratee reputation is being determined, the lower the weighting of the rating. Optionally, each rating may be placed in a temporal order according to the time at which the rating was provided, and each rating may be weighted according its position in this order. Further, time may be divided into a number of intervals, and the temporal intervals may be placed in a temporal order. A rating then may be weighted according to the position of the temporal interval in the temporal order.
- Similar to as described above in relation to determining a rater reputation, temporally weighting each rating represents the principle that an entity's ratee reputation may change over time. Thus, more recent ratings should be given more weight than older ratings in determining the ratee reputation of an entity. Accordingly, a ratee reputation may be determined by applying the following equation:
where ƒ(j) is a temporal function such as, for example,Equation 12, above. Other weightings and combinations of weightings may be applied to a rating to determine a ratee reputation of an entity. - Determining a ratee reputation of an entity by averaging ratings may become more cost prohibitive as the number of ratings provided by other entities for the first entity increases.
- Accordingly, in another embodiment of determining the ratee reputation of an entity, the ratee reputation may be determined recursively. Similarly to as described above in relation to determining a rater reputation, determining the ratee reputation recursively saves computational space and time, particularly as the number of ratings provided for the rated entity increases.
- A recursive determination may include: providing a previously determined ratee reputation (i.e., an initial ratee reputation) of the entity; determining a ratee reputation adjustment (which may be a positive or negative value) based on a received rating, and adding the reputation adjustments to the previously determined rater reputation.
-
FIG. 8 is a flow chart illustrating an example implementation of a method of recursively determining a ratee reputation of an entity. InAct 90, an expected rating may be determined from the range and the ratee's initial reputation. An expected rating may be determined by applyingEquation 4 or a variation thereof. - Next, in
Act 92, the expected rating may be subtracted from the first rating to produce a ratee reputation modification. The ratee reputation modification may be produced as described in relation to Equation 23 below. - Next, in
Act 94, the ratee reputation modification may be scaled by a rater reputation of the rating entity to produce a scaled ratee reputation modification. This rater reputation may be the resulting rater reputation generated according to one of the methods described in relation toFIGS. 2 and 3 , or a variation thereof. - Combining
Acts
R i scaled =R i rater·(W i −E i),Equation 24
where Ri rater is the rater reputation, Wi is the first rating, Ei is the expected rating, and Wi-Ei is the ratee reputation modification. - Next, in
Act 96, the scaled ratee reputation modification may be damped by a damping factor to produce a damped ratee reputation modification. The damped ratee reputation modification may be produced by applying the following equation:
where
is the initial ratee reputation, and
is the damping factor. Optionally, the damping factor may be determined (in series or concurrently with Acts 90-94) by applyingEquation 2, where the acceleration factor, a, may be predetermined to control the rate at which the ratee reputation can change. - In a
next Act 98, the damped ratee reputation modification may be divided by a change rate factor to produce a ratee reputation adjustment. The change rate factor affects the rate at which the ratee reputation may change. - Next, in
Act 100, the ratee reputation may be added to the initial ratee reputation to produce a resulting ratee reputation. In accordance withActs
where C is the change rate factor and Ri ratee is the resulting ratee reputation. - In an aspect of recursively determining a ratee reputation of an entity using one or more rater reputations, for example, by applying
Equation 26, an original ratee reputation, R0 rater, for the entity has its lowest value before any ratings are provided for the entity. Assigning a lowest value to an original ratee reputation prevents an entity with a low ratee reputation from creating a new identity as a new entity and beginning with a higher ratee reputation. - Other methods of determining a ratee reputation may be used.
- A ratee reputation of a first entity may be used for a variety of purposes, including determining whether to transact with the first entity, determining a price to pay for a good or service of the first entity and determining a price to pay for insuring a quality of a good or service of the first entity.
-
FIG. 9 is a data flow diagram illustrating an example embodiment of asystem 109 for generating a resultingratee reputation 140. Aratee reputation generator 110 may receive anacceleration factor 112, arange 32, an initialratee reputation 114, afirst rating 116, arater reputation 118 and achange rate factor 120, and generate a resultingratee reputation 140 as output. - The
ratee reputation generator 110 may include a dampingfactor generator 130, an expectedrating generator 122, a rateereputation modification generator 126, a rateereputation adjustment generator 134 and anadder 138. - The expected
rating generator 122 may receive as input therange 32 and the initialratee reputation 114, and generate an expectedrating 124, for example, by applyingEquation 3. The rateereputation modification generator 126 may receive thefirst rating 116 and the expectedrating 124, and generate theratee reputation modification 128, for example, as described above in relation to Equation 23. - The damping
factor generator 130 may receive theacceleration factor 112, the initialratee reputation 114 and therange 32, and generate the dampingfactor 132, for example, by applyingEquation 3 above. - The ratee
reputation adjustment generator 134 may receive theratee reputation modification 128, the dampingfactor 132, therater reputation 118 and thechange rate factor 120 and generate theratee reputation adjustment 136, for example, by applying Equation 25 above. - The
adder 138 may receive the initialratee reputation 114 and theratee reputation adjustment 136 and generate the resultingratee reputation 140, for example, by applyingEquation 26 above. - Although not shown in
FIG. 9 , thesystem 109 also may include a transaction module such astransaction module 528 described below in relation toFIG. 11 . The transaction module may receive the resultingratee reputation 140, determine whether to transact with the rated entity based on the resulting ratee reputation, and then output a value, for example, a boolean value, which indicates whether or not to transact with the rated entity. - The
ratee reputation generator 110, and any combination of itscomponents ratee reputation generator 110 and any components thereof may reside on a single machine (e.g., a computer), or may be modular and reside on multiple interconnected (e.g., by a network) machines. Further, on each of the one or machines that include theratee reputation generator 110 or a component thereof, thegenerator 110 or component may reside in one or more locations on the machine. For example, different portions of theratee reputation generator 110 or different portions of a component may reside in different areas of memory (e.g., RAM, ROM, disk, etc.) on a computer. - The
acceleration factor 112, therange 32 and thechange rate factor 120 may be stored as constants in a reputation database or similar data structure as described below in relation toFIG. 18 . The initialratee reputation 114 and therater reputation 118 may also be stored in the reputation database or similar data structure. In response to receiving thefirst rating 116, the ratee reputation generator may access the reputation database and retrievevalues ratee reputation 140 as described above. The resultingratee reputation 140 then may be stored in the reputation database or similar structure for later access. -
System 109, and components thereof, are merely example embodiments of a system for generating a ratee reputation. Such example embodiments are not meant to limit the scope of the invention and are provided merely for illustrative purposes, as any of a variety of other systems and components for determining a ratee reputation may fall within the scope of the invention. - V. Determining a Reputation of an Entity From a Perspective of Another Entity
-
FIG. 10 is a flowchart illustrating an example embodiment of a method for determining a personalized ratee reputation. In afirst Act 402, a breadth-first search is performed beginning at the second entity to determine, from one or more rating paths, one or more first rating paths that have a first length. In anext Act 404, for each determined first rating path, a third entity that has a level equal to or less than the first length is identified. - In a
next Act 406, for each identified third party, a first rating of the first identity provided by the third entity is determined. For each identified third entity, it may be determined that the third entity has provided more than one rating of the first entity. For each third entity for which it has been determined that more than rating has been provided, a most recent rating may be selected from the one or more ratings to serve as the first rating of the first entity provided by the third entity. - In a following
Act 408, the first ratings are combined. The first ratings may be combined in any of a variety of ways. In one embodiment, an average of the first ratings is calculated. Calculating this first average may include, for each first rating, weighting the first rating as a function of a personalized ratee reputation of the corresponding third entity from the prospective of the second entity. This weighting may be relative to personalized ratee reputations of the other third entities from the perspective of the second entity. - The personalized ratee reputation of the third entity may be determined in any of a variety of ways. In one embodiment, one or more fourth entities that are on the first rating paths, that have provided a second rating of the third entity and that have a level equal to one less than the level of the third entity are determined. The second ratings provided by these one or more fourth entities are then combined to produce the personalized ratee reputation of the third entity from the perspective of the second entity. Combining the second ratings may include calculating an average of the second ratings. Optionally,
Equation 4 or a variation thereof may be applied to combine the ratings. - In a
next Act 410, the personalized ratee reputation is produced by weighting the combined first rating as a function of the first length. Act 410 may be performed by application of the following Equation:
where Rk(n) is the personalized ratee reputation of an entity k from a perspective of a second entity a distance n from the entity k, Wjk(n) is a rating provided by an entity j for the entity k, where the entity j is a distance n−1 from the second entity, Rj(n−1) is the personalized ratee reputation of the entity j from the perspective of the second entity, D is a range of allowable personalized ratee reputation valves, and f(n) is a function of the distances n between the second entity and the entity K (i.e., a function of the length of the rating paths between the second entity and entity K), such as, for example:
where T is a constant number having a value>1. - This personalized ratee reputation may then be used to determine whether to transact with the first entity.
- Weighting the personalized ratee reputation as a function of the distance between the first and second entity, where the shorter distance the greater the value of the weighting represents the principle that the less attenuated the rating path between a first and second entity, the more likely the second entity is to trust the determined personalized ratee reputation.
- A personalized ratee reputation of a first entity may be used for a variety of purposes, including determining whether to transact with the first entity, determining a price to pay for a good or service of the first entity and determining a price to pay for insuring a quality of a good or service of the first entity.
-
FIG. 11 is a data flow diagram illustrating an example embodiment of asystem 500 for determining a personalized ratee reputation of a first entity from the perspective of a second entity. Thesystem 500 may include a personalizedratee reputation generator 506 and atransaction module 528. - The personalized
ratee reputation generator 506 may include a path-searchingmodule 508, a first look-upmodule 514, aprocessing module 512, a second look-upmodule 520 and aratings combiner 524. - The path-searching
module 508 may receive an indication of thefirst entity 502, an indication of thesecond entity 504 andother ratings 517, and generate a signal indicatingfirst rating paths 510. The path-searching module may determine thefirst rating paths 510 as described above in relation to Act 402 ofFIG. 10 . The path-searchingmodule 508 may determine thefirst rating path 510 by searching or looking-up rating values in a ratings database or reputation database as described in more detail below in connection toFIG. 18 . - The first look-up
module 514 may receive the indication of thefirst rating paths 510 and generatethird entities 518, for example, as described above in connection toAct 404. The first look-upmodule 514 may determine thethird entities 518 by looking up the third entities in a ratings or reputation database as described in more detail in relation toFIG. 18 . - The second look-up
module 520 may receive the indication of thethird entity 518 as input and generate as output thefirst ratings 522, for example, as described above in relation toAct 406. The second look-upmodule 520 may determine thefirst ratings 522 by looking up the first ratings in a ratings or reputation database, using thethird entity 518 as a look-up criteria, as described in more detail below in relation toFIG. 14 . - The
processing module 512 receives the indication of thefirst rating paths 510 and the indication of thethird entities 518 and generatessecond ratings 516. Theprocessing module 512 may generate thesecond ratings 516 as described above in relation toAct 410. Theprocessing module 512 may determine thesecond ratings 516 by looking up values in the ratings or reputation database described below in relation toFIG. 14 , using thefirst rating paths 510 and thethird entities 518 as search criteria. - The
ratings combiner 524 receives thefirst ratings 522 and thesecond ratings 516 and generates the personalizedratee reputation 526. The ratings combiner 524 may also receiveother ratings 517, which may have been generated in a similar manner to thefirst ratings 522 and thesecond ratings 516, for example, by applying Equation 27. The ratings combiner 524 may generate the personalizedratee reputation 526 as described above in relation toAct 410, and then may store the personalized ratee reputation in the reputation database or similar structure for later access. - The
transaction module 528 may receive the personalizedratee reputation 526, determine whether to transact with the first entity based on the personalized ratee reputation, and then output a value, for example, aBoolean value 530, which indicates whether or not to transact with the first entity. Theboolean rating 530 then may be stored in the reputation database or similar structure for later access. - The
transaction module 528, thepersonalized reputation generator 506 and any component thereof, including thepath searching module 508, thefirst lookup module 514, thesecond lookup module 520, theprocessing module 512 and theratings combiner 524, may be implemented as software, hardware, firmware or any combination thereof. Themodule 528,generator 506 and components thereof may reside on a single machine (e.g., a computer), or may be modular and reside on multiple interconnected (e.g., by a network) machines. Further, on each of the one or more machines that include thetransaction module 528,personalized reputation generator 506 or a component thereof, thegenerator 506,transaction module 528 or component may reside in one or more locations on the machine. For example, different portions of thepersonalized reputation generator 506, different portions of a component, and different portions of thetransaction module 528 may reside in different areas of memory (e.g., RAM, ROM, disk, etc.) on a computer. -
System 500, and components thereof, are merely example embodiments of a system for generating a personalized reputation generator and transacting based on thepersonalized reputation generator 506. Such example embodiments are not meant to limit the scope of the invention and are provided merely for illustrative purposes, as any of a variety of other systems and components for generating a personalized ratee reputation and transacting based thereon may fall within the scope of the invention. - VI. Determining a Temporal Estimate of the Impact of One or More Attributes on an Overall Attribute or Result
- The methods and systems described above for determining a rater reputation, a rater reputation deviation, a ratee reputation, a ratee reputation deviation and a personalized ratee reputation are all described in the context of a single rating being provided for each exchange between a rating entity and a rated entity.
- Alternatively, in an aspect of rating an exchange, multiple ratings may be provided. For example, in an exchange involving the sale of a good, the rating entity may provide a rating for the quality of the good, a rating for the service quality, a rating for the promptness of delivery and an overall rating for the exchange. If such multiple ratings are received for a single exchange, the systems and methods described above for determining reputations and deviations may be applied to each rating independently. For example, the rater reputation and rater reputation deviation of a rating entity as a rater of the quality of the good may be determined or the ratee reputation and ratee reputation deviation of a rated entity for the promptness of its delivery of goods may be determined.
- Further, if multiple ratings are received for a single exchange, and an overall rating is provided or a result based on the transaction occurs, it may be desirable to determine the impact of the one or more ratings on the overall rating or the result. It further may be desirable to determine the impact of these one or more ratings on an overall rating or results over time as ratings are received.
- Accordingly, a method and system for recursively estimating the impact of one or more attributes, e.g., ratings, on a result corresponding to a qualitative assessment, e.g., an overall rating, where the qualitative assessment is based on the one or more attributes, will now be described. Determining such a recursive estimate over time allows the impacts of one or more ratings to be incrementally corrected as more ratings are received.
- Although recursively estimating the impact of one or more attributes on a result is described below mostly in the context of ratings, the impacts of other types of attributes may be used. For example, one or more objective metrics may be assigned according to the quality of a good, the time of delivery, the cost of the good, etc., and the impact of this metric on a result or qualitative assessment may be determined.
- A result of a qualitative assessment may be the qualitative assessment itself such as, for example, feedback, a rating or a score, or may be an action taken as a result of a qualitative assessment such as setting a price or making a selection. For example, if several entities purchase a good from a seller, and each entity provides separate ratings for the quality of the good, the service quality and the promptness of delivery, and also provides an overall rating of the seller, it may be desirable to determine the impact of each rating on the overall rating. Subsequently, the impact of each rating may be applied to estimate the impact of future ratings on a future overall rating.
- It may be desirable to determine recursively a temporal estimate of the impact of one or more attributes of a plurality of attributes in any of the following scenarios:
- a) Determining an overall rating;
- b) Setting a price of a narrowly defined service or item;
- c) Determining a frequency (i.e., a selectivity ratio) that a particular entity, item or service is selected when compared with other entities items or services; and
- d) Determining any other rating aspect, score, or other quality assessment for an entity, item, service, or policy that can be reduced to a numerical or Boolean state.
-
FIG. 12 is a flow chart illustrating an example embodiment of amethod 800 for estimating the impacts of a plurality of ratings on a result. In afirst Act 802, a plurality of ratings provided for a first entity by a second entity in association with an exchange between the first and second entity are received. Each of these ratings corresponds to a respective rating parameter for the first entity. For example, rating parameters may include quality of service, speed of delivery and quality of a good, and the corresponding ratings may be 0.5, 0.7 and 0.2, respectively. - In a
next Act 804, a result based on the exchange between the first and second entity may be received. This result may correspond to a result parameter for the first entity. For example, the result parameter may be an overall rating or a price, and the corresponding result may be 0.7 or $7.95, respectively. - In a following
Act 806, for each received rating, a corresponding initial weighting value that corresponds to the rating parameter of the received rating is accessed. This initial weighting represents an estimated weight of the received rating for the rating parameter relative to received ratings for the other rating parameters in determining the result for the result parameter. For example, historically, the received ratings for the quality of a good sold by a first entity may have more weight in determining a price of the good than ratings received for the speed of delivery. For example, the speed of delivery may have a corresponding initial weighting value of 0.6, and the quality of the good may have a corresponding initial weighting value of 0.4. - Next, in
Act 808, an estimated result is determined by applying each initial weighting value to its corresponding receive rating. For example, if a rating of 0.6 is received for the speed of delivery and the rating of 0.4 is received for the quality of the good, and the speed of delivery has an initial weighting value 0.3 and the quality of the good has an initial weighting value of 0.7, then the estimated result may be determined as (0.6)(0.3)+(0.4)(0.7)=0.46. - In a
next Act 810, the estimated result may be compared to the received results to produce a weighting modification. For example, if the estimated result is 0.46, but the actual received result is 0.054, then the weighting modification may be 0.54−0.46=0.08. - In a following
Act 812, for each initial weighting value, the initial weighting value may be adjusted as a function of the weighting modification to produce a resulting weighting value for the corresponding rating parameter. For example, the weighting modification may be added to each initial weighting value. - In an aspect of generating a resulting weighting value, the initial weighting values may be adjusted as a function of previous ratings provided for the rating parameters, for example, as described in more detail below in connection to
FIG. 14 . -
FIG. 13 is a data flow diagram illustrating an example embodiment of a system for estimating the impacts of a plurality of ratings on a result. Thesystem 169 may include aratings database 205 and aweightings generator 170 that includes aweighting modification generator 194 and aweighting adjuster 195. Theratings database 205 may be a reputation database or similar data structure such as described below in relation toFIG. 18 . - The
weighting modification generator 194 may receiveratings 175 and aresult 178, access initial weighting values 179 corresponding to theratings 175, and generate theweighting modification 196, as described above in Acts 802-810 ofFIG. 12 . Theratings 175 may be received along with afirst entity ID 177 that identifies the first entity for which theratings 175 are being provided. Theweighting modification generator 194 may access the initial weighting values 179 from theratings database 205 using thefirst entity ID 177. In an embodiment, theweighting modification generator 194 may generate theweighting modification 196 as described above in relation to Act 156 ofFIG. 14 . - The
weighting adjuster 195 may receive theweighting modification 196 and the initial weighting values 179, and generate the resulting weighting values 203 as described above in relation toAct 812. In an embodiment, theweighting adjuster 195 may generate the resulting weighting values as described below in relation toActs FIG. 14 . - The
weighting adjuster 194 may store the resulting weighting values 203 and theratings database 205 for later access. - In aspects of determining recursively the impacts of two or more ratings on a result, the initial weighting values and resulting weighting values may be represented as weighting vectors or weighting matrices. Further, the plurality of provided ratings may be represented by a rating vector or rating matrix, as will be described in more detail below. Such vectors and matrices may be embodied tangibly on a medium using any of a variety of data structures, for example, an array. Accordingly, the terms “weighting array” and “rating array,” as used herein, refer to a weighting vector or matrix and a rating vector or matrix, respectively, tangibly embodied on a medium as an array of other data structure.
-
FIG. 14 is a flow chart illustrating an example embodiment of a method of estimating the impacts of a plurality of ratings on a result using vectors. InAct 150, a multi-rating vector is transposed to produce a transposed multi-rating vector. Each value of the multi-rating vector may represent an attribute or sub-rating of an overall rating or other qualitative assessment. - Next, in
Act 152, a resulting adjustment matrix may be generated from an initial adjustment matrix, the multi-rating vector and the transposed multi-rating vector. The resulting adjustment matrix may be generated by applying the following equation:
V t =F·V t−1 +X t X t T, Equation 29
where Vt−1 is an initial adjustment matrix, Xt is the multi-rating vector, Xt T is the transposed multi-rating vector, and Vt is the resulting adjustment matrix. As indicated inEquation 26, the initial adjustment matrix may be weighted by a forgetting factor, F, similar to as described above in relation toEquations 18 and 19. The forgetting factor, F, may have the affect of giving the multi-rating vector more weight than the initial adjustment matrix in determining the resulting adjustment matrix. Alternatively, the forgetting factor may be chosen such that the initial adjustment matrix has more weight in determining the resulting adjustment matrix than the multi-rating matrix. - In a
next Act 154, the resulting adjustment matrix may be inverted to produce an inverted adjustment matrix. In a followingAct 156, a weighting modification may be generated from the transposed multi-rating vector, an initial weighting vector and a result (e.g., an overall rating or other qualitative assessment). The weighting modification may be generated by applying the following equation:
Z mod =Y t −X t T Z t−12,Equation 30
where Yt−1 is the initial weighting vector, Yt is the result, and Zmod is the weighting modification. The weighting modification represents a difference between the result Yt and an estimated result, Xt TZt−1, according to the initial weighting vector and the multi-rating vector. - Next, in
Act 158, a weighting adjustment vector may be generated from the inverted adjustment matrix, the multi-rating vector and the weighting modification. Such a weighting adjustment vector may be generated by applying the following equation:
Z adj =V t −1 X t ·Z mod Equation 31
where Vt −1 is the inverted adjustment matrix and Zadj is the weighting adjustment vector. - The weighting adjustment, Zadj further may be weighted as a function of any number of factors. For example, the weighting adjustment may be weighted as a function of the reputation of the rater that provided the multi-rating vector and overall result, or as a function of some other factor corresponding to the transaction on which the multi-rating vector and the overall result are based, e.g., a factor indicative of the significance of the transaction.
- Next, in
Act 160, the weighting adjustment vector may be added to the initial weighting vector to produce a resulting weighting vector. The resulting weighting vector may be generated by applying the following equation:
Z t =Z t−1 +Z adj,Equation 32
where Zt is the resulting weighting vector. - In an aspect of using vectors to determine weighting values, each weighting vector includes an extra weighting value such that if there is a number of rating parameters n, then the dimension of the weighting vector is n+1. This extra weighting value may be used to factor in a parameter other than a rating parameter in determining a result parameter.
- For the embodiment of
FIG. 14 , estimated impacts of a plurality of ratings are determined for a single result. In such an embodiment, weighting vectors are applied to rating vectors and compared to a single scalar result. - Alternatively, more than one result may correspond to a plurality of ratings. If more than one result is involved, the plurality of results may be represented with a vector, and the impacts of the plurality of ratings on the plurality of results may be represented by a weighting matrix, as opposed to a weighting vector. The use of weighting matrices and a result vector as opposed to weighting vectors and a result scalar are intended to fall within the scope of the invention. Weighting vectors and result scalars are described for illustrative purposes and are not intended to limit the scope of the invention.
- Equations 29-32 are derived from the minimization of the following recursive least squares (RLS) problem:
where F(t) is the constant F described above in relation toFIG. 14 . - The above RLS problem, the theory behind the problem, and the several matrix manipulations needed to solve
equation 30 to produce equations 26-29 are described in more detail inchapter 9 of the Madsen text. - Other statistical methods for determining recursively a weighting vector for multiple attributes of a qualitative assessment, for example, variations of
Equation 32, may be used. - In an aspect of determining a plurality of weighting values indicating an impact of a plurality of ratings on a result, a estimated result deviation may be determined. In an embodiment of determining a estimated result deviation, the estimated result deviation may be determined recursively. For example, the estimated result deviation may be determined by application of the following equation:
where RDi weight is the resulting estimated result deviation,
is an initial estimated result deviation, F is a forgetting factor similar to as described above in relation toEquations 18 and 19, and To is the effective number of estimated results similar to as described above in relation toEquations 18 and 19. - Other statistical methods may be used to determine reliability of estimated results based on the estimated impacts of a plurality of ratings on a result. For example, a forgetting factor may not be applied, or an average deviation may be determined for all estimated deviations for the rated entity, or such an average deviation may be calculated with each estimated result deviation weighted according to how recent the estimated result was determined. For example, the estimated result deviation may be determined by applying the following equation:
-
FIG. 15 is a data flow diagram illustrating an example embodiment of thesystem 169 that generates a resultingweighting vector 204 to represent the estimated weights of a plurality of ratings in determining a result. Theweighting generator 170 may receive as input aninitial adjustment matrix 174, amulti-rating matrix 176, an overall rating (or other qualitative assessment) 178 and aninitial weighting vector 180, and generate as output a resultingweighting vector 204. - The
weighting generator 170 may include anadjustment matrix generator 182, aninverse matrix generator 190, a transposedvector generator 186, theweighting modification generator 194 and theweighting adjuster 195. Theweighting adjuster 195 includes aweighting adjustment generator 198 and a vector adder 202. - The transposed
vector generator 186 may receive as input themulti-rating vector 176 and generate the transposedmulti-rating vector 184 as output. Theadjustment matrix generator 182 may receive as input the transposedmulti-rating vector 184 and theinitial adjustment matrix 174 and generate as output a resultingadjustment matrix 188. Theadjustment matrix generator 182 may also receive the forgettingfactor 172 as input and generate the resultingadjustment matrix 188 based at least in part on the forgettingfactor 172. The resultingadjustment matrix 188 may be generated by applying Equation 29. - The
inverse matrix generator 190 may receive as input a resultingadjustment matrix 188 and generate as output aninverse adjustment matrix 192. Theweighting modification generator 194 may receive as input the transposedmulti-rating vector 184 and theoverall rating 178 and generate as output aweighting modification 196. Optionally, theweighting modification generator 194 may generate the weighting modification in accordance withEquation 30 above. - The
weighting adjustment generator 198 may receive as input theweighting modification 196, themulti-rating vector 176 and theinverse adjustment matrix 192, and generate as output theweighting adjustment vector 200. Theweighting adjustment generator 198 may generate theweighting adjustment vector 200 in accordance withEquation 31 above. - The vector adder 202 may receive as input the
weighting adjustment vector 200 and theinitial weighting vector 180 and generate as output the resultingweighting vector 204. The vector adder 202 may generate the resultingweighting vector 204 in accordance withEquation 32. - The
weighting generator 170 and any components thereof, includingadjustment matrix generator 182,inverse matrix generator 190, transposedvector generator 186,weighting modification generator 194weighting adjuster 195,weighting adjustment generator 198 and vector adder 202, may be implemented as software, hardware, firmware or any combination thereof. Theweighting vector generator 170 and any components thereof may reside on a single machine (e.g., a computer), or may be modular and reside on multiple interconnected (e.g., by a network) machines. Further, on each of the one or more machines that include theweighting vector generator 169 or a component thereof, thegenerator 170 or component may reside in one or more locations on the machine. For example, different portions of theweighting vector generator 170 or different portions of a component may reside in different areas of memory (e.g., RAM, ROM, disk, etc.) on a computer. - The
system 169, and components thereof are merely example embodiments of a system for generating a ratee reputation. Such example embodiments are not meant to limit the scope of the invention and are provided merely for illustrative purposes, as any of a variety of other systems and components for estimating the impacts of a plurality of ratings on a result may fall within the scope of the invention. - For any of the methods or systems described herein in relation to
FIGS. 2-17 , any number of entities may be members of a population. This population may be grouped into clusters having common properties, for example, similar demographics or preferences. Further, for a transaction having multiple parameters, for example, multiple rating parameters, the entities of the population may be grouped into any number of clusters based on the statistical similarity of the ratings they provide for the parameters. The entities may be grouped in to clusters using any of a number of known clustering algorithms, for example, a k-means clustering algorithm, an example of which is described at: http://www.cne.gmu.edu/modules/dau/stat/clustgalgs/clust5_bdy.html. - If a population of entities is grouped into clusters, reputations, estimators, deviations, and other values may be determined for each cluster separately. For example, weighting vectors and associated matrices may be generated and maintained independently for each cluster.
- VII. Generating a Third Estimated Ratee Reputation By Combining a First Estimated Ratee Reputation With a Second Estimated Ratee Reputation
- Described above are example embodiments of systems and methods for estimating a ratee reputation. For example, determining an estimated ratee reputation of a first entity using the rater reputations of one or more entities that provided ratings for the first entity is described above in relation to
FIGS. 8 and 9 . - As described above, if one or more attributes are provided for a single exchange with a ratee (e.g., one or more ratings are provided by a rater), an estimated ratee reputation may be determined for each attribute. As used herein, an estimated ratee attribute reputation is an estimated ratee reputation determined for one of multiple attributes for a single exchange, as opposed to an estimated ratee reputation determined for a result or qualitative assessment associated with the single exchange.
- Further, determining an estimated result by applying initial weighting values to associated attributes, e.g., received ratings, is described above in relation to Act 808 of
FIG. 12 . If the estimated result described in relation toAct 808 is an overall rating, then initial weighting values may be applied to associated attributes to estimate a ratee reputation of an entity. - It may be desirable to combine two or more estimated ratee reputations to produce a new estimated ratee reputation. The deviation of such a new estimated ratee reputation over time from actual received ratings may be used to determine whether the new estimated ratee reputation is more accurate than the two or more estimated ratee reputations. Accordingly, provided herein is a method and system for estimating a ratee reputation of a first entity based on multiple estimated ratee reputations of the first entity.
-
FIG. 16 is a flowchart illustrating an example embodiment of amethod 600 of combining a first estimated ratee reputation of a first entity with a second estimated ratee reputation of the first entity to produce a third estimated ratee reputation. A reputation database, for example, thereputation database 726 described below in relation toFIG. 17 , may store a first estimated ratee reputation and one or more ratee attribute reputations. - The first estimated ratee reputation may have been generated by combining a plurality of overall ratings, where each overall rating was provided by a different entity for a corresponding transaction. For example, the first estimated ratee reputation may be determined from the plurality of overall ratings in a similar manner to that described above in relation to
FIGS. 8 and 9 . - Each of the one or more ratee attribute reputations may have been generated by combining a plurality of attributes, where each attribute corresponds to a different one of the transactions. For example, each of the one or more ratee attribute reputations may be determined from the plurality of attributes in a similar manner to that described above in relation to
FIGS. 8 and 9 . - In
Act 602, the one or more ratee attribute reputations are received. In anext Act 604, for each received ratee attribute reputation, a corresponding weighting value is accessed. The accessed weighting value represents an estimated weight of the received ratee attribute reputation relative to others of the one or more received ratee attribute reputations in estimating a ratee reputation of the first entity, similarly to as described above in relation to Act 806 ofFIG. 12 . - Next, in
Act 606, a second estimated ratee reputation of the first entity is determined by applying each weighting value to its corresponding ratee attribute reputation. For example, the weighting values may be combined with the corresponding ratee attribute reputations similarly to as described above in relation to Act 808 ofFIG. 12 andAct 156 ofFIG. 14 . - In a following
Act 608, the first estimated ratee reputation is accessed, for example, from a database. Act 608 may be performed serially or concurrently to the sequence of Acts 602-606. - Next, in
Act 610, the first estimated ratee reputation and the second estimated ratee reputation may be combined to produce a third estimated ratee reputation. For example, the average of the first estimated ratee reputation and the second estimated ratee reputation may be determined such that the third estimated ratee reputation is an average of the first and second estimated ratee reputations. - Optionally, the first and second estimated ratee reputations may be weighted by estimated ratee reputation deviations to calculate a weighted average. For example, to determine the third estimated ratee reputation, the following equation may be applied:
where R1 is the first estimated ratee reputation, R2 is second estimated ratee reputation and R3 is the third estimated ratee reputation. RD1 rater is a first estimated ratee reputation deviation corresponding to the first estimated ratee reputation and may be determined by application ofEquation 21 as described above. RD2 rater is a second estimated ratee reputation deviation corresponding to the second estimated ratee reputation and may be determined by application ofEquation 34 or 35 as described above. - As described above in relation to
Equations - Accordingly, in Equation 36, to weight both the first and second ratee reputations according to their respective reliabilities, the first ratee reputation is weighted by the second ratee reputation deviation, and the second ratee reputation is weighted by the first ratee reputation deviation. As a result, of the first and second ratee reputations, the ratee reputation having the higher reliability will have the greater impact on the third estimated ratee reputation.
-
FIG. 17 is a data flow diagram illustrating an example embodiment of asystem 700 for generating an estimated ratee reputation. Thesystem 700 may include areputation database 726 and an estimatedratee reputation generator 706 that includes a firstratee reputation estimator 708 and a secondratee reputation estimator 712. Thereputation database 726 may be a reputation database or similar data structure as described below in relation toFIG. 18 . - The estimated
ratee reputation generator 706 may receiveratee attribute reputations 702 and aratee ID 704. Theratee attribute reputations 702 may be determined, as described above, from attributes corresponding to transactions with a first entity, and theratee ID 704 may indicate the first entity. The estimatedratee reputation generator 706 may use theratee ID 704 to access, fromreputation database 726, weighting values 718, first estimatedratee reputation deviation 720, second estimatedratee reputation 722 and second estimatedratee reputation deviation 724. The estimatedratee reputation generator 706 may usevalues ratee reputation 714, for example, as described above in relation toFIG. 16 . - The first
ratee reputation estimator 708 may receive the one or moreratee attribute reputations 702 and the weighting values 718, and generate the second estimatedratee reputation 710, for example, as described above in relation to Act 606 of theFIG. 16 . - The second
ratee reputation estimator 712 may receive the first estimatedratee reputation deviation 720, the second estimatedratee reputation 722, the second estimatedratee reputation deviation 724 and the second estimatedratee reputation 710, and generate the third estimatedratee reputation 714, for example, as described above in relation to Act 610 ofFIG. 16 . - The estimated
ratee reputation generator 706, and any of a combination of its components, including firstratee reputation estimator 708 and secondratee reputation estimator 712, may be implemented using software, firmware or hardware or any combination thereof. The estimatedratee reputation generator 706 and any components thereof may reside on a single machine (e.g., a computer), or may be modular and reside on a multiple interconnected (e.g., by a network) machines. Further, on each of the one or more machines that include the estimatedratee reputation generator 706 or a component thereof, thegenerator 706 or component may reside in one or more locations on the machine. For example, different portions of the estimatedratee reputation generator 706 or different portions of a component may reside in different areas of memory (e.g., RAM, ROM, disk, etc.) on a computer. -
System 700, and components thereof, are merely example embodiments of a system for estimating a ratee reputation. Such example embodiments are not meant to limit the scope of the invention and are provided merely for illustrative purposes, as any of a variety of other system and components for estimating a ratee reputation may fall within the scope of the invention. - VIII. System Architecture
-
FIG. 18 is a data flow diagram illustrating anexample system architecture 209 for implementing the methods, systems and variations thereof described above in relation toFIGS. 2-17 . Thesystem 209 may include aclient 210, aserver 212,ratee reputation database 234, an authentication I.D.database 236 and arater reputation database 238. Thecomponents system 209 may have a variety of configurations. For example, all these components may reside on a single computer, or any combination thereof may reside on a separate computer or multiple computers interconnected, for example, by a network. Further, any combination of these components may reside on separate networks, including separate LANs (Local Area Networks), MANs (Metropolitan Area Networks) and WANs (Wide Area Networks). - The
ratee reputation database 234 may contain one or more entries, where each entry represents ratee reputation data for an entity. For example, an entry in the ratee reputation database may include an entity I.D., demographic information about the entity, ratings of the entity provided by other entities, a ratee reputation of the entity, a ratee reputation deviation of the entity, weighting values corresponding to attributes or rating parameters or items, goods, services, people, etc. associated with the entity, and other information regarding the entity. - The authentication I.D.
database 236 may include one or more entries, where each entry represents an entity that is part of a population using a ratee reputation system, a rater reputation system, or a combination thereof. An entry in the authentication I.D.database 236 may include an I.D. of an entity, demographic information about an entity, a password for the entity, and other information regarding an entity. - The
rater reputation database 238 may include one or more entries, where each entry represents data specific to an entity. For example, an entry in therater reputation database 238 may include an entity I.D., demographic information about the entity, ratings provided by the entity, a rater reputation of the entity, a rater reputation deviation of the entity, and other information regarding the entity. - Each of the
databases databases - The
server 212 may include afrontend 230 and abackend 232. Thefrontend 230 may include a user interface and may receive and send instructions and other data to and from theclient 210. Thefrontend 230 may then transfer and receive instructions to and from thebackend 232. - The backend may determine ratee reputations, rater reputations, ratee reputation deviations, rater reputation deviations, weightings, and other metrics and values such as, for example, as described above in relation to
FIGS. 2-17 . - The
backend 232 may also receivequeries 216 submitted to thefrontend 230 and translate these queries to database queries on one of thedatabases backend 232 may also transfer updatedreputations 244,ratings 246 and weightings to the various databases. - The
client 210 and theserver 212 may be implemented as software, firmware, or hardware, or any combination thereof. For example, theclient 210 and theserver 230 may be implemented as software using any of a variety of programming languages, such as Java, C and C++. - Through the
client 210, an entity may submit one ormore ratings 224 to theserver 212. Included along with theratings 224 may be a rater ID that identifies the rating entity and a ratee ID that identifies the entity being rated. Theserver 212 may then determine a ratee reputation, a rater reputation, weightings, and other information in accordance with the various techniques described above in relation toFIGS. 2-17 . - For example, to determine a ratee reputation of the rated entity, the
server 212 may submit aratee query 240 to theratee reputation database 234 and extract various data corresponding to the rated entity, including the current ratee reputation and previous ratings of the rated entity provided by other entities. This data may be sent from theratee reputation database 234 to theserver 212 as query results 242. Theserver 212 then may use the query results 242 and the receivedratings 224 to determine a ratee reputation of the rated entity according to any of a variety of the techniques described above , then may send theratee reputation 226 through thefrontend 230 to theclient 210. - The
server 212 may store or persist the one ormore ratings 224 and the updatedratee reputation 244 by sending to theratee reputation database 234, for example, to an entry corresponding to the rated entity, and by sending the one ormore ratings 224 and the updated ratee reputation to therater reputation database 238, for example, to an entry corresponding to the rating entity. - Similarly, to determine a rater reputation of the rating entity, the
server 212 may submit arater query 248 to therater reputation database 238 and extract various data corresponding to the rating entity, including the current rater reputation and previous ratings of other entities provided by the rating entity. This data may be sent from therater reputation database 238 to theserver 212 as query results 250. Theserver 212 then may use the query results 250 and the receivedratings 224 to determine a rater reputation of the rated entity according to any of the techniques described above through thefrontend 230, theserver 212, then may send the determinedrater reputation 226 to theclient 210. - The
server 212 may store the one ormore ratings 224 and the updatedrater reputation 244 by sending to theratee reputation database 234, for example, to an entry corresponding to the rated entity, and by sending the one ormore ratings 224 and the updatedrater reputation 244 to therater reputation database 238, for example, to an entry corresponding to the rating entity. - The
server 212 may use similar techniques to access theratee reputation database 234 and therater reputation database 238 to determine other values as described above in relation toFIGS. 2-17 , including determining ratee reputation deviations, rater reputation deviations, personalized ratee reputations, weightings, estimated results, and estimated result deviations. - For example, in response to receiving a plurality of
ratings 224 and a result from theclient 210, theserver 212 may access the necessary values in theratee reputation database 234 and/or theratee reputation database 238, determine resulting weighting values, for example, as described above in relation toFIGS. 12-15 , and then sending the weighting values 214 to theclient 210. - Further, the
server 212, through thefrontend 230, may receive auser query 216 from theclient 210. For example, the user query may be requesting the rater reputations of one or more entities, the ratee reputations of one or more entities, or the personalized ratee reputation of one or more entities from the perspective of a particular entity. Theserver 212 may convert theuser query 216 into a database query, for example, aratee query 240 or arater query 248, and send the database query to the appropriate database. Theserver 212 then may send the query results 218 to theclient 210. - The
server 212 may be part of an on-line marketplace, for example, an agent-mediated marketplace. Accordingly, the client may request and receivemarketplace information 220 from theserver 230. Further, theserver 212, as part of a transaction between an entity corresponding to theclient 210 and a counterpart entity, may sendcommunications 222 to theclient 210. Thecommunications 222 may include notifications pertaining to the current transaction, prompts for information from the entity corresponding to theclient 210, reputations of the counterpart entity, and other information about the entity including demographic data, weighting values, etc. - During the context of an exchange between the entity corresponding to the
client 210 and the counterpart entity, theclient 210 may sendinstructions 228 to make a deal, for example, instructions to purchase a good. - The
client 210 may include a user interface to allow interaction between a user and an application, for example, a reputation application or marketplace application implemented using theclient 210 andserver 212. The user interface may involve using CGI scripts to generate web pages in accordance with any of a variety of markup languages such as, for example, HTML, XML or SGML. - The several methods described herein, and the various embodiments thereof, including the methods and embodiments described in relation to
FIGS. 2, 3 , 8, 10, 12, 14, 16 and Equations 5-35, may be stored on a computer-readable medium as computer-readable signals. For each method or embodiment thereof, the computer-readable signals may define instructions that, in response to being executed on a computer, perform the method or embodiment. Each method or embodiment thereof may be implemented using software, firmware, hardware, or any combination thereof. If implemented as software, the computer-readable signals defining the method or embodiment may be part of a computer program product, such as a software program written in any of a number of languages such as, for example, C, C++ or Java. - Having now described some illustrative embodiments, it should be apparent to those skilled in the art that the foregoing is merely illustrative and not limiting, having been presented by way of example only. Numerous modifications and other illustrative embodiments are within the scope of one of ordinary skill in the art and are contemplated as falling within the scope of the invention. In particular, although many of the examples presented herein involve specific combinations of method Acts or apparatus elements, it should be understood that those Acts and those elements may be combined in other ways to accomplish the same objectives. Acts, elements and features discussed only in connection with one embodiment are not intended to be excluded from a similar role in other embodiments.
Claims (1)
1. A method of determining a rater reputation of a first entity as a rater of other entities, wherein provided is a first rating indicative of a rating of a rated entity by the first entity, and one or more second ratings, each second rating indicative of a rating of the rated entity provided by another entity, the method comprising acts of:
(A) combining the second ratings to produce a first combined rating;
(B) comparing the first rating to the first combined rating to produce a first rating predictability of the first rating, the first rating predictability being a negative function of a magnitude of a difference between the first rating and the first combined rating; and
(C) generating a resulting rater reputation based at least in part on the first rating predictability.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/056,889 US20050149383A1 (en) | 2000-06-02 | 2005-02-11 | Method and system for ascribing a reputation to an entity as a rater of other entities |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US20905900P | 2000-06-02 | 2000-06-02 | |
US09/710,008 US6895385B1 (en) | 2000-06-02 | 2000-11-10 | Method and system for ascribing a reputation to an entity as a rater of other entities |
US11/056,889 US20050149383A1 (en) | 2000-06-02 | 2005-02-11 | Method and system for ascribing a reputation to an entity as a rater of other entities |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/710,008 Continuation US6895385B1 (en) | 2000-06-02 | 2000-11-10 | Method and system for ascribing a reputation to an entity as a rater of other entities |
Publications (1)
Publication Number | Publication Date |
---|---|
US20050149383A1 true US20050149383A1 (en) | 2005-07-07 |
Family
ID=34576232
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/710,008 Expired - Fee Related US6895385B1 (en) | 2000-06-02 | 2000-11-10 | Method and system for ascribing a reputation to an entity as a rater of other entities |
US11/056,889 Abandoned US20050149383A1 (en) | 2000-06-02 | 2005-02-11 | Method and system for ascribing a reputation to an entity as a rater of other entities |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/710,008 Expired - Fee Related US6895385B1 (en) | 2000-06-02 | 2000-11-10 | Method and system for ascribing a reputation to an entity as a rater of other entities |
Country Status (1)
Country | Link |
---|---|
US (2) | US6895385B1 (en) |
Cited By (46)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050065810A1 (en) * | 2003-09-05 | 2005-03-24 | France Telecom | Evaluation of reputation of an entity by a primary evaluation centre |
US20060224442A1 (en) * | 2005-03-31 | 2006-10-05 | Round Matthew J | Closed loop voting feedback |
US20070033092A1 (en) * | 2005-08-04 | 2007-02-08 | Iams Anthony L | Computer-implemented method and system for collaborative product evaluation |
US20070179834A1 (en) * | 2006-02-01 | 2007-08-02 | Novell, Inc. | Federation and attestation of online reputations |
US20070204325A1 (en) * | 2006-02-24 | 2007-08-30 | Microsoft Corporation | Personal identification information schemas |
US20070203852A1 (en) * | 2006-02-24 | 2007-08-30 | Microsoft Corporation | Identity information including reputation information |
US20070204168A1 (en) * | 2006-02-24 | 2007-08-30 | Microsoft Corporation | Identity providers in digital identity system |
US20080028215A1 (en) * | 2006-07-28 | 2008-01-31 | Microsoft Corporation | Portable personal identity information |
US20080046446A1 (en) * | 2006-08-21 | 2008-02-21 | New York University | System, method, software arrangement and computer-accessible medium for incorporating qualitative and quantitative information into an economic model |
US20080133657A1 (en) * | 2006-11-30 | 2008-06-05 | Havoc Pennington | Karma system |
US20080178272A1 (en) * | 2007-01-18 | 2008-07-24 | Microsoft Corporation | Provisioning of digital identity representations |
US20080178271A1 (en) * | 2007-01-18 | 2008-07-24 | Microsoft Corporation | Provisioning of digital identity representations |
US20080177691A1 (en) * | 2007-01-24 | 2008-07-24 | Secure Computing Corporation | Correlation and Analysis of Entity Attributes |
US20080184339A1 (en) * | 2007-01-26 | 2008-07-31 | Microsoft Corporation | Remote access of digital identities |
US20090138325A1 (en) * | 2002-01-08 | 2009-05-28 | Britt Jr Joe Freeman | Apparatus and method for identifying and surveying subscribers |
US20090144226A1 (en) * | 2007-12-03 | 2009-06-04 | Kei Tateno | Information processing device and method, and program |
US20090144272A1 (en) * | 2007-12-04 | 2009-06-04 | Google Inc. | Rating raters |
US20090157491A1 (en) * | 2007-12-12 | 2009-06-18 | Brougher William C | Monetization of Online Content |
US7779156B2 (en) | 2007-01-24 | 2010-08-17 | Mcafee, Inc. | Reputation based load balancing |
US20100257184A1 (en) * | 2006-12-20 | 2010-10-07 | Victor David Uy | Method and apparatus for scoring electronic documents |
US7937480B2 (en) * | 2005-06-02 | 2011-05-03 | Mcafee, Inc. | Aggregation of reputation data |
US20120053996A1 (en) * | 2010-08-31 | 2012-03-01 | Frankmon Group, S.R.O. | System and method for objective performance evaluation in employment recruiting process |
US8214497B2 (en) | 2007-01-24 | 2012-07-03 | Mcafee, Inc. | Multi-dimensional reputation scoring |
US8250012B1 (en) * | 2007-03-30 | 2012-08-21 | Amazon Technologies, Inc. | Evaluating recommendations by determining user actions, and performance values pertaining to lists of recommendations |
US20130018877A1 (en) * | 2007-01-31 | 2013-01-17 | Reputation.com | Identifying and Changing Personal Information |
US8549611B2 (en) | 2002-03-08 | 2013-10-01 | Mcafee, Inc. | Systems and methods for classification of messaging entities |
US8561167B2 (en) | 2002-03-08 | 2013-10-15 | Mcafee, Inc. | Web reputation scoring |
US8578480B2 (en) | 2002-03-08 | 2013-11-05 | Mcafee, Inc. | Systems and methods for identifying potentially malicious messages |
US8589503B2 (en) | 2008-04-04 | 2013-11-19 | Mcafee, Inc. | Prioritizing network traffic |
US8621559B2 (en) | 2007-11-06 | 2013-12-31 | Mcafee, Inc. | Adjusting filter or classification control settings |
US8621638B2 (en) | 2010-05-14 | 2013-12-31 | Mcafee, Inc. | Systems and methods for classification of messaging entities |
US8635690B2 (en) | 2004-11-05 | 2014-01-21 | Mcafee, Inc. | Reputation based message processing |
US20140156556A1 (en) * | 2012-03-06 | 2014-06-05 | Tal Lavian | Time variant rating system and method thereof |
US8763114B2 (en) | 2007-01-24 | 2014-06-24 | Mcafee, Inc. | Detecting image spam |
WO2014109781A1 (en) * | 2013-01-13 | 2014-07-17 | Qualcomm Incorporated | Improving user generated rating by machine classification of entity |
TWI463422B (en) * | 2011-01-17 | 2014-12-01 | Inventec Appliances Corp | Action recording system |
US20150012546A1 (en) * | 2006-02-17 | 2015-01-08 | Google Inc. | Sharing user distributed search results |
US9047327B2 (en) | 2012-12-03 | 2015-06-02 | Google Technology Holdings LLC | Method and apparatus for developing a social hierarchy |
US9378287B2 (en) | 2011-12-14 | 2016-06-28 | Patrick Frey | Enhanced search system and method based on entity ranking |
US9875313B1 (en) * | 2009-08-12 | 2018-01-23 | Google Llc | Ranking authors and their content in the same framework |
WO2018018132A1 (en) * | 2016-07-29 | 2018-02-01 | 1974226 Alberta Ltd. | Processing user provided information for ranking information modules |
US10878017B1 (en) | 2014-07-29 | 2020-12-29 | Groupon, Inc. | System and method for programmatic generation of attribute descriptors |
US10909585B2 (en) | 2014-06-27 | 2021-02-02 | Groupon, Inc. | Method and system for programmatic analysis of consumer reviews |
US10977667B1 (en) | 2014-10-22 | 2021-04-13 | Groupon, Inc. | Method and system for programmatic analysis of consumer sentiment with regard to attribute descriptors |
US11250450B1 (en) * | 2014-06-27 | 2022-02-15 | Groupon, Inc. | Method and system for programmatic generation of survey queries |
US20230040330A1 (en) * | 2018-04-24 | 2023-02-09 | Torre Labs, Inc. | Professional social networking services, methods and systems |
Families Citing this family (66)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7433832B1 (en) | 1999-11-19 | 2008-10-07 | Amazon.Com, Inc. | Methods and systems for distributing information within a dynamically defined community |
US6895385B1 (en) * | 2000-06-02 | 2005-05-17 | Open Ratings | Method and system for ascribing a reputation to an entity as a rater of other entities |
US20040073617A1 (en) | 2000-06-19 | 2004-04-15 | Milliken Walter Clark | Hash-based systems and methods for detecting and preventing transmission of unwanted e-mail |
US7356484B2 (en) * | 2000-10-03 | 2008-04-08 | Agile Software Corporation | Self-learning method and apparatus for rating service providers and predicting future performance |
US7080117B2 (en) * | 2000-11-17 | 2006-07-18 | Robert dePinto | System and method for exchanging creative content |
JP2002183324A (en) * | 2000-12-11 | 2002-06-28 | Dokonjo Soft:Kk | Www server on internet for enabling browse or evaluation of information provided on proposed theme and executing point control of user following proposal, provision, browse and evaluation, and operation method of server |
US7428496B1 (en) * | 2001-04-24 | 2008-09-23 | Amazon.Com, Inc. | Creating an incentive to author useful item reviews |
JP3834485B2 (en) * | 2001-05-28 | 2006-10-18 | 株式会社日立製作所 | LASER DRIVE DEVICE, OPTICAL DISK DEVICE USING SAME, AND LASER CONTROL METHOD THEREOF |
US7464059B1 (en) | 2001-09-21 | 2008-12-09 | Yt Acquisition Corporation | System and method for purchase benefits at a point of sale |
US9189788B1 (en) | 2001-09-21 | 2015-11-17 | Open Invention Network, Llc | System and method for verifying identity |
US7269737B2 (en) | 2001-09-21 | 2007-09-11 | Pay By Touch Checking Resources, Inc. | System and method for biometric authorization for financial transactions |
US7437330B1 (en) | 2002-09-20 | 2008-10-14 | Yt Acquisition Corp. | System and method for categorizing transactions |
WO2003034637A2 (en) * | 2001-10-18 | 2003-04-24 | Transpose, Llc | System and method for measuring rating reliability through rater prescience |
US7693947B2 (en) * | 2002-03-08 | 2010-04-06 | Mcafee, Inc. | Systems and methods for graphically displaying messaging traffic |
US7458098B2 (en) * | 2002-03-08 | 2008-11-25 | Secure Computing Corporation | Systems and methods for enhancing electronic communication security |
US7694128B2 (en) * | 2002-03-08 | 2010-04-06 | Mcafee, Inc. | Systems and methods for secure communication delivery |
US8132250B2 (en) | 2002-03-08 | 2012-03-06 | Mcafee, Inc. | Message profiling systems and methods |
US7096498B2 (en) | 2002-03-08 | 2006-08-22 | Cipher Trust, Inc. | Systems and methods for message threat management |
US7124438B2 (en) * | 2002-03-08 | 2006-10-17 | Ciphertrust, Inc. | Systems and methods for anomaly detection in patterns of monitored communications |
US7870203B2 (en) | 2002-03-08 | 2011-01-11 | Mcafee, Inc. | Methods and systems for exposing messaging reputation to an end user |
US7903549B2 (en) | 2002-03-08 | 2011-03-08 | Secure Computing Corporation | Content-based policy compliance systems and methods |
JP2005536816A (en) * | 2002-08-19 | 2005-12-02 | チョイスストリーム インコーポレイテッド | Statistical specific personal recommendation system |
US8332263B2 (en) * | 2002-12-19 | 2012-12-11 | Oracle International Corporation | System and method for configuring scoring rules and generating supplier performance ratings |
US20040153358A1 (en) * | 2003-01-31 | 2004-08-05 | Lienhart Deborah A. | Method and system for prioritizing user feedback |
US7599938B1 (en) | 2003-07-11 | 2009-10-06 | Harrison Jr Shelton E | Social news gathering, prioritizing, tagging, searching, and syndication method |
US8554601B1 (en) | 2003-08-22 | 2013-10-08 | Amazon Technologies, Inc. | Managing content based on reputation |
US7822631B1 (en) * | 2003-08-22 | 2010-10-26 | Amazon Technologies, Inc. | Assessing content based on assessed trust in users |
JP2005135071A (en) * | 2003-10-29 | 2005-05-26 | Hewlett-Packard Development Co Lp | Method and device for calculating trust values on purchase |
US20050261926A1 (en) * | 2004-05-24 | 2005-11-24 | Hartridge Andrew J | System and method for quantifying and communicating a quality of a subject entity between entities |
US20060009994A1 (en) * | 2004-07-07 | 2006-01-12 | Tad Hogg | System and method for reputation rating |
JP2006091971A (en) * | 2004-09-21 | 2006-04-06 | Hewlett-Packard Development Co Lp | Network data display method/device/program |
US20060116970A1 (en) * | 2004-11-18 | 2006-06-01 | Helmut Scherzer | System and method to grant or refuse access to a system |
US8175726B2 (en) | 2005-01-24 | 2012-05-08 | Microsoft Corporation | Seeding in a skill scoring framework |
WO2006104534A2 (en) * | 2005-03-25 | 2006-10-05 | The Motley Fool, Inc. | Scoring items based on user sentiment and determining the proficiency of predictors |
US20060217994A1 (en) * | 2005-03-25 | 2006-09-28 | The Motley Fool, Inc. | Method and system for harnessing collective knowledge |
US7519562B1 (en) * | 2005-03-31 | 2009-04-14 | Amazon Technologies, Inc. | Automatic identification of unreliable user ratings |
US20060272002A1 (en) * | 2005-05-25 | 2006-11-30 | General Knowledge Technology Design | Method for automating the management and exchange of digital content with trust based categorization, transaction approval and content valuation |
CA2613200A1 (en) * | 2005-06-28 | 2007-01-04 | Choicestream, Inc. | Methods and apparatus for a statistical system for targeting advertisements |
CA2614440C (en) | 2005-07-07 | 2016-06-21 | Sermo, Inc. | Method and apparatus for conducting an information brokering service |
US8015484B2 (en) * | 2006-02-09 | 2011-09-06 | Alejandro Backer | Reputation system for web pages and online entities |
US8374973B2 (en) * | 2006-02-16 | 2013-02-12 | Microsoft Corporation | Reputation system |
US8489438B1 (en) * | 2006-03-31 | 2013-07-16 | Intuit Inc. | Method and system for providing a voice review |
EP1855245A1 (en) * | 2006-05-11 | 2007-11-14 | Deutsche Telekom AG | A method and a system for detecting a dishonest user in an online rating system |
US8131559B2 (en) * | 2006-11-17 | 2012-03-06 | National Ict Australia Limited | Accepting documents for publication or determining an indication of the quality of documents |
US8843385B2 (en) * | 2006-12-11 | 2014-09-23 | Ecole Polytechnique Federale De Lausanne (Epfl) | Quality of service monitoring of a service level agreement using a client based reputation mechanism encouraging truthful feedback |
JP2008158712A (en) * | 2006-12-22 | 2008-07-10 | Fujitsu Ltd | Evaluation information management method, evaluation information management system and evaluation information management program |
US8175989B1 (en) | 2007-01-04 | 2012-05-08 | Choicestream, Inc. | Music recommendation system using a personalized choice set |
US8179798B2 (en) | 2007-01-24 | 2012-05-15 | Mcafee, Inc. | Reputation based connection throttling |
US8402517B2 (en) * | 2007-06-20 | 2013-03-19 | Microsoft Corporation | Content distribution and evaluation providing reviewer status |
US8548996B2 (en) * | 2007-06-29 | 2013-10-01 | Pulsepoint, Inc. | Ranking content items related to an event |
US20090006115A1 (en) * | 2007-06-29 | 2009-01-01 | Yahoo! Inc. | Establishing and updating reputation scores in online participatory systems |
US20090070460A1 (en) * | 2007-09-12 | 2009-03-12 | Ebay Inc. | Method and system for social network analysis |
US20090070130A1 (en) * | 2007-09-12 | 2009-03-12 | Neelakantan Sundaresan | Reputation scoring |
US8045458B2 (en) | 2007-11-08 | 2011-10-25 | Mcafee, Inc. | Prioritizing network traffic |
US10083420B2 (en) | 2007-11-21 | 2018-09-25 | Sermo, Inc | Community moderated information |
US8160975B2 (en) | 2008-01-25 | 2012-04-17 | Mcafee, Inc. | Granular support vector machine with random granularity |
US20090198562A1 (en) * | 2008-01-31 | 2009-08-06 | Guenter Wiesinger | Generating social graph using market data |
US20090265229A1 (en) * | 2008-04-18 | 2009-10-22 | Ranbir S Sidhu | System, method, and program product for buyer driven services e-commerce |
US9009082B1 (en) | 2008-06-30 | 2015-04-14 | Amazon Technologies, Inc. | Assessing user-supplied evaluations |
US20110066497A1 (en) * | 2009-09-14 | 2011-03-17 | Choicestream, Inc. | Personalized advertising and recommendation |
US8364544B2 (en) | 2010-06-18 | 2013-01-29 | Prairie Pacific Holdings, LLC | Comprehensive online bidding and sales management system for merchant processing services |
US9202200B2 (en) | 2011-04-27 | 2015-12-01 | Credibility Corp. | Indices for credibility trending, monitoring, and lead generation |
US8719203B2 (en) * | 2012-02-17 | 2014-05-06 | Cisco Technology, Inc. | Method and apparatus for time-based decay of knowledge base ratings |
US9507831B1 (en) * | 2015-12-29 | 2016-11-29 | International Business Machines Corporation | Credibility enhancement for online comments and recommendations |
US20190050917A1 (en) * | 2017-08-14 | 2019-02-14 | ScoutZinc, LLC | System and method for rating of enterprise using crowdsourcing in combination with weighted evaluator ratings |
US11816622B2 (en) * | 2017-08-14 | 2023-11-14 | ScoutZinc, LLC | System and method for rating of personnel using crowdsourcing in combination with weighted evaluator ratings |
Citations (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4870579A (en) * | 1987-10-01 | 1989-09-26 | Neonics, Inc. | System and method of predicting subjective reactions |
US4996642A (en) * | 1987-10-01 | 1991-02-26 | Neonics, Inc. | System and method for recommending items |
US5550746A (en) * | 1994-12-05 | 1996-08-27 | American Greetings Corporation | Method and apparatus for storing and selectively retrieving product data by correlating customer selection criteria with optimum product designs based on embedded expert judgments |
US5732400A (en) * | 1995-01-04 | 1998-03-24 | Citibank N.A. | System and method for a risk-based purchase of goods |
US5768142A (en) * | 1995-05-31 | 1998-06-16 | American Greetings Corporation | Method and apparatus for storing and selectively retrieving product data based on embedded expert suitability ratings |
US6073117A (en) * | 1997-03-18 | 2000-06-06 | Kabushiki Kaisha Toshiba | Mutual credit server apparatus and a distributed mutual credit system |
US6112186A (en) * | 1995-06-30 | 2000-08-29 | Microsoft Corporation | Distributed system for facilitating exchange of user information and opinion using automated collaborative filtering |
US6249785B1 (en) * | 1999-05-06 | 2001-06-19 | Mediachoice, Inc. | Method for predicting ratings |
US6275811B1 (en) * | 1998-05-06 | 2001-08-14 | Michael R. Ginn | System and method for facilitating interactive electronic communication through acknowledgment of positive contributive |
US20010034631A1 (en) * | 2000-01-21 | 2001-10-25 | Kiselik Daniel R. | Method and apparatus for the automatic selection of parties to an arrangement between a requestor and a satisfier of selected requirements |
US6347332B1 (en) * | 1999-12-30 | 2002-02-12 | Edwin I. Malet | System for network-based debates |
US6362837B1 (en) * | 1997-05-06 | 2002-03-26 | Michael Ginn | Method and apparatus for simultaneously indicating rating value for the first document and display of second document in response to the selection |
US6487541B1 (en) * | 1999-01-22 | 2002-11-26 | International Business Machines Corporation | System and method for collaborative filtering with applications to e-commerce |
US6708155B1 (en) * | 1999-07-07 | 2004-03-16 | American Management Systems, Inc. | Decision management system with automated strategy optimization |
US6892179B1 (en) * | 2000-06-02 | 2005-05-10 | Open Ratings Inc. | System and method for ascribing a reputation to an entity |
US6892178B1 (en) * | 2000-06-02 | 2005-05-10 | Open Ratings Inc. | Method and system for ascribing a reputation to an entity from the perspective of another entity |
US6895385B1 (en) * | 2000-06-02 | 2005-05-17 | Open Ratings | Method and system for ascribing a reputation to an entity as a rater of other entities |
-
2000
- 2000-11-10 US US09/710,008 patent/US6895385B1/en not_active Expired - Fee Related
-
2005
- 2005-02-11 US US11/056,889 patent/US20050149383A1/en not_active Abandoned
Patent Citations (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4870579A (en) * | 1987-10-01 | 1989-09-26 | Neonics, Inc. | System and method of predicting subjective reactions |
US4996642A (en) * | 1987-10-01 | 1991-02-26 | Neonics, Inc. | System and method for recommending items |
US5550746A (en) * | 1994-12-05 | 1996-08-27 | American Greetings Corporation | Method and apparatus for storing and selectively retrieving product data by correlating customer selection criteria with optimum product designs based on embedded expert judgments |
US5732400A (en) * | 1995-01-04 | 1998-03-24 | Citibank N.A. | System and method for a risk-based purchase of goods |
US5768142A (en) * | 1995-05-31 | 1998-06-16 | American Greetings Corporation | Method and apparatus for storing and selectively retrieving product data based on embedded expert suitability ratings |
US6112186A (en) * | 1995-06-30 | 2000-08-29 | Microsoft Corporation | Distributed system for facilitating exchange of user information and opinion using automated collaborative filtering |
US6073117A (en) * | 1997-03-18 | 2000-06-06 | Kabushiki Kaisha Toshiba | Mutual credit server apparatus and a distributed mutual credit system |
US6362837B1 (en) * | 1997-05-06 | 2002-03-26 | Michael Ginn | Method and apparatus for simultaneously indicating rating value for the first document and display of second document in response to the selection |
US6275811B1 (en) * | 1998-05-06 | 2001-08-14 | Michael R. Ginn | System and method for facilitating interactive electronic communication through acknowledgment of positive contributive |
US6487541B1 (en) * | 1999-01-22 | 2002-11-26 | International Business Machines Corporation | System and method for collaborative filtering with applications to e-commerce |
US6249785B1 (en) * | 1999-05-06 | 2001-06-19 | Mediachoice, Inc. | Method for predicting ratings |
US6708155B1 (en) * | 1999-07-07 | 2004-03-16 | American Management Systems, Inc. | Decision management system with automated strategy optimization |
US6347332B1 (en) * | 1999-12-30 | 2002-02-12 | Edwin I. Malet | System for network-based debates |
US20010034631A1 (en) * | 2000-01-21 | 2001-10-25 | Kiselik Daniel R. | Method and apparatus for the automatic selection of parties to an arrangement between a requestor and a satisfier of selected requirements |
US6892179B1 (en) * | 2000-06-02 | 2005-05-10 | Open Ratings Inc. | System and method for ascribing a reputation to an entity |
US6892178B1 (en) * | 2000-06-02 | 2005-05-10 | Open Ratings Inc. | Method and system for ascribing a reputation to an entity from the perspective of another entity |
US6895385B1 (en) * | 2000-06-02 | 2005-05-17 | Open Ratings | Method and system for ascribing a reputation to an entity as a rater of other entities |
Cited By (73)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090138325A1 (en) * | 2002-01-08 | 2009-05-28 | Britt Jr Joe Freeman | Apparatus and method for identifying and surveying subscribers |
US8135609B2 (en) * | 2002-01-08 | 2012-03-13 | Microsoft Corporation | Identifying and surveying subscribers |
US8549611B2 (en) | 2002-03-08 | 2013-10-01 | Mcafee, Inc. | Systems and methods for classification of messaging entities |
US8561167B2 (en) | 2002-03-08 | 2013-10-15 | Mcafee, Inc. | Web reputation scoring |
US8578480B2 (en) | 2002-03-08 | 2013-11-05 | Mcafee, Inc. | Systems and methods for identifying potentially malicious messages |
US20050065810A1 (en) * | 2003-09-05 | 2005-03-24 | France Telecom | Evaluation of reputation of an entity by a primary evaluation centre |
US7769594B2 (en) * | 2003-09-05 | 2010-08-03 | France Telecom | Evaluation of reputation of an entity by a primary evaluation centre |
US8635690B2 (en) | 2004-11-05 | 2014-01-21 | Mcafee, Inc. | Reputation based message processing |
US8566144B2 (en) * | 2005-03-31 | 2013-10-22 | Amazon Technologies, Inc. | Closed loop voting feedback |
US20060224442A1 (en) * | 2005-03-31 | 2006-10-05 | Round Matthew J | Closed loop voting feedback |
US7937480B2 (en) * | 2005-06-02 | 2011-05-03 | Mcafee, Inc. | Aggregation of reputation data |
US8249915B2 (en) * | 2005-08-04 | 2012-08-21 | Iams Anthony L | Computer-implemented method and system for collaborative product evaluation |
US20070033092A1 (en) * | 2005-08-04 | 2007-02-08 | Iams Anthony L | Computer-implemented method and system for collaborative product evaluation |
US20070179834A1 (en) * | 2006-02-01 | 2007-08-02 | Novell, Inc. | Federation and attestation of online reputations |
US20150012546A1 (en) * | 2006-02-17 | 2015-01-08 | Google Inc. | Sharing user distributed search results |
US20070203852A1 (en) * | 2006-02-24 | 2007-08-30 | Microsoft Corporation | Identity information including reputation information |
US8104074B2 (en) | 2006-02-24 | 2012-01-24 | Microsoft Corporation | Identity providers in digital identity system |
US8117459B2 (en) * | 2006-02-24 | 2012-02-14 | Microsoft Corporation | Personal identification information schemas |
US20070204168A1 (en) * | 2006-02-24 | 2007-08-30 | Microsoft Corporation | Identity providers in digital identity system |
US20070204325A1 (en) * | 2006-02-24 | 2007-08-30 | Microsoft Corporation | Personal identification information schemas |
US8078880B2 (en) | 2006-07-28 | 2011-12-13 | Microsoft Corporation | Portable personal identity information |
US20080028215A1 (en) * | 2006-07-28 | 2008-01-31 | Microsoft Corporation | Portable personal identity information |
US7848979B2 (en) | 2006-08-21 | 2010-12-07 | New York University | System, method, software arrangement and computer-accessible medium for incorporating qualitative and quantitative information into an economic model |
US20080046446A1 (en) * | 2006-08-21 | 2008-02-21 | New York University | System, method, software arrangement and computer-accessible medium for incorporating qualitative and quantitative information into an economic model |
US20080133657A1 (en) * | 2006-11-30 | 2008-06-05 | Havoc Pennington | Karma system |
US8386564B2 (en) * | 2006-11-30 | 2013-02-26 | Red Hat, Inc. | Methods for determining a reputation score for a user of a social network |
US20100257184A1 (en) * | 2006-12-20 | 2010-10-07 | Victor David Uy | Method and apparatus for scoring electronic documents |
US9390173B2 (en) * | 2006-12-20 | 2016-07-12 | Victor David Uy | Method and apparatus for scoring electronic documents |
US20080178271A1 (en) * | 2007-01-18 | 2008-07-24 | Microsoft Corporation | Provisioning of digital identity representations |
US8087072B2 (en) | 2007-01-18 | 2011-12-27 | Microsoft Corporation | Provisioning of digital identity representations |
US20080178272A1 (en) * | 2007-01-18 | 2008-07-24 | Microsoft Corporation | Provisioning of digital identity representations |
US8407767B2 (en) | 2007-01-18 | 2013-03-26 | Microsoft Corporation | Provisioning of digital identity representations |
US7779156B2 (en) | 2007-01-24 | 2010-08-17 | Mcafee, Inc. | Reputation based load balancing |
US9544272B2 (en) | 2007-01-24 | 2017-01-10 | Intel Corporation | Detecting image spam |
US8762537B2 (en) | 2007-01-24 | 2014-06-24 | Mcafee, Inc. | Multi-dimensional reputation scoring |
US8214497B2 (en) | 2007-01-24 | 2012-07-03 | Mcafee, Inc. | Multi-dimensional reputation scoring |
US9009321B2 (en) | 2007-01-24 | 2015-04-14 | Mcafee, Inc. | Multi-dimensional reputation scoring |
US10050917B2 (en) | 2007-01-24 | 2018-08-14 | Mcafee, Llc | Multi-dimensional reputation scoring |
US7949716B2 (en) | 2007-01-24 | 2011-05-24 | Mcafee, Inc. | Correlation and analysis of entity attributes |
US20080177691A1 (en) * | 2007-01-24 | 2008-07-24 | Secure Computing Corporation | Correlation and Analysis of Entity Attributes |
US8578051B2 (en) | 2007-01-24 | 2013-11-05 | Mcafee, Inc. | Reputation based load balancing |
US8763114B2 (en) | 2007-01-24 | 2014-06-24 | Mcafee, Inc. | Detecting image spam |
US20080184339A1 (en) * | 2007-01-26 | 2008-07-31 | Microsoft Corporation | Remote access of digital identities |
US9521131B2 (en) | 2007-01-26 | 2016-12-13 | Microsoft Technology Licensing, Llc | Remote access of digital identities |
US8689296B2 (en) | 2007-01-26 | 2014-04-01 | Microsoft Corporation | Remote access of digital identities |
US20130018877A1 (en) * | 2007-01-31 | 2013-01-17 | Reputation.com | Identifying and Changing Personal Information |
US8250012B1 (en) * | 2007-03-30 | 2012-08-21 | Amazon Technologies, Inc. | Evaluating recommendations by determining user actions, and performance values pertaining to lists of recommendations |
US8762310B2 (en) | 2007-03-30 | 2014-06-24 | Amazon Technologies, Inc. | Evaluating recommendations |
US8621559B2 (en) | 2007-11-06 | 2013-12-31 | Mcafee, Inc. | Adjusting filter or classification control settings |
US20090144226A1 (en) * | 2007-12-03 | 2009-06-04 | Kei Tateno | Information processing device and method, and program |
US20090144272A1 (en) * | 2007-12-04 | 2009-06-04 | Google Inc. | Rating raters |
US9760547B1 (en) | 2007-12-12 | 2017-09-12 | Google Inc. | Monetization of online content |
US20090157491A1 (en) * | 2007-12-12 | 2009-06-18 | Brougher William C | Monetization of Online Content |
US8150842B2 (en) | 2007-12-12 | 2012-04-03 | Google Inc. | Reputation of an author of online content |
US8606910B2 (en) | 2008-04-04 | 2013-12-10 | Mcafee, Inc. | Prioritizing network traffic |
US8589503B2 (en) | 2008-04-04 | 2013-11-19 | Mcafee, Inc. | Prioritizing network traffic |
US9875313B1 (en) * | 2009-08-12 | 2018-01-23 | Google Llc | Ranking authors and their content in the same framework |
US8621638B2 (en) | 2010-05-14 | 2013-12-31 | Mcafee, Inc. | Systems and methods for classification of messaging entities |
US20120053996A1 (en) * | 2010-08-31 | 2012-03-01 | Frankmon Group, S.R.O. | System and method for objective performance evaluation in employment recruiting process |
TWI463422B (en) * | 2011-01-17 | 2014-12-01 | Inventec Appliances Corp | Action recording system |
US9378287B2 (en) | 2011-12-14 | 2016-06-28 | Patrick Frey | Enhanced search system and method based on entity ranking |
US20140156556A1 (en) * | 2012-03-06 | 2014-06-05 | Tal Lavian | Time variant rating system and method thereof |
US9047327B2 (en) | 2012-12-03 | 2015-06-02 | Google Technology Holdings LLC | Method and apparatus for developing a social hierarchy |
US9311347B2 (en) | 2012-12-03 | 2016-04-12 | Google Technology Holdings LLC | Method and apparatus for developing a social hierarchy |
WO2014109781A1 (en) * | 2013-01-13 | 2014-07-17 | Qualcomm Incorporated | Improving user generated rating by machine classification of entity |
US10909585B2 (en) | 2014-06-27 | 2021-02-02 | Groupon, Inc. | Method and system for programmatic analysis of consumer reviews |
US11250450B1 (en) * | 2014-06-27 | 2022-02-15 | Groupon, Inc. | Method and system for programmatic generation of survey queries |
US10878017B1 (en) | 2014-07-29 | 2020-12-29 | Groupon, Inc. | System and method for programmatic generation of attribute descriptors |
US11392631B2 (en) | 2014-07-29 | 2022-07-19 | Groupon, Inc. | System and method for programmatic generation of attribute descriptors |
US10977667B1 (en) | 2014-10-22 | 2021-04-13 | Groupon, Inc. | Method and system for programmatic analysis of consumer sentiment with regard to attribute descriptors |
WO2018018132A1 (en) * | 2016-07-29 | 2018-02-01 | 1974226 Alberta Ltd. | Processing user provided information for ranking information modules |
US11386173B2 (en) | 2016-07-29 | 2022-07-12 | 1974226 Alberta Ltd. | Processing user provided information for ranking information modules |
US20230040330A1 (en) * | 2018-04-24 | 2023-02-09 | Torre Labs, Inc. | Professional social networking services, methods and systems |
Also Published As
Publication number | Publication date |
---|---|
US6895385B1 (en) | 2005-05-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6895385B1 (en) | Method and system for ascribing a reputation to an entity as a rater of other entities | |
US6892179B1 (en) | System and method for ascribing a reputation to an entity | |
US6892178B1 (en) | Method and system for ascribing a reputation to an entity from the perspective of another entity | |
US8635098B2 (en) | Determining a community rating for a user using feedback ratings of related users in an electronic environment | |
US7711606B2 (en) | Method and computer readable medium for assisting a customer in choosing among commodities | |
US8826022B1 (en) | Methods and systems for creating monetary accounts for members in a social network | |
US20060059141A1 (en) | System and method for providing information via a network | |
US20110047044A1 (en) | Method and Apparatus for Evaluating Fraud Risk in an Electronic Commerce Transaction | |
US20120197762A1 (en) | System and method for identifying an alternative provider of telecommunications services | |
US20090164274A1 (en) | Variable learning rate automated decisioning | |
US20080201373A1 (en) | Evaluator credibility based content credibility evaluation apparatus and method thereof | |
US7848953B2 (en) | Order fulfillment logic for a field service system | |
US8190732B1 (en) | Method and apparatus for providing information associated with service providers using a social network | |
JP2003150812A (en) | Dealing mediation service providing method and system | |
US20060036497A1 (en) | Decision theoretic approach to targeted solicitation by maximizing expected profit increases | |
Polat et al. | Privacy‐preserving top‐N recommendation on distributed data | |
US10021150B2 (en) | Systems and methods of establishing and measuring trust relationships in a community of online users | |
WO2001095211A2 (en) | System and method for recursively estimating a reputation of an entity | |
Bamasak et al. | A distributed reputation management scheme for mobile agent based e-commerce applications | |
WO2015191741A1 (en) | Systems and methods for conducting relationship dependent online transactions | |
KR100918167B1 (en) | Method for studying user profile using user inclination data | |
AU2021107316A4 (en) | Systems and methods for data filtering | |
AU2021102386A4 (en) | Systems and methods for data filtering | |
Kravari et al. | T-rex: A hybrid agent trust model based on witness reputation and personal experience | |
Shourya et al. | Collaborative Filtering Techniques for Recency Frequency Monetary Analysis |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |