Sample · 3 tasks · 1 NPC

Contract Clause Evaluation

Frontier models graded against a written rubric, with explicit obligations, with verdicts that say fit for use or not fit for use.

Clause

Preliminary Provisions

Definitions

Intepretation variation 1

fail · strict fail

Coverage

75%

Verdict

FAIL

Confidence

0.97

Tokens

211 / 4,884

in / out

Prompt · Candidate · Rubric

Prompt

Act as an experienced international contracts lawyer. Draft a comprehensive "Interpretation" clause for a commercial agreement in English.

Candidate · claude-opus-4-7 (max)

[●].    INTERPRETATION

[●].1    In this Agreement, unless the context otherwise requires:

(a)    the headings and sub-headings to the Sections, Schedules, Annexes and Exhibits are inserted for convenience of reference only and shall not affect the construction or interpretation of this Agreement;

(b)    words denoting the singular include the plural and vice versa, and words denoting any gender include all genders;

(c)    the words "include", "includes" and "including" shall be construed as if followed by the words "without limitation", and shall not be construed as limiting any general statement which they follow to the specific or similar items or matters immediately following them;

(d)    the word "person" includes any individual, firm, partnership, body corporate, joint venture, association, trust, unincorporated organisation, governmental, regulatory or supranational authority, or any other entity (in each case whether or not having separate legal personality), and includes any successor in title to, or permitted assignee of, the foregoing;

(e)    any reference to a "party" or the "parties" is a reference to a party or the parties to this Agreement and includes their respective successors in title and permitted assignees;

(f)    any reference to a Section, Schedule, Annex or Exhibit is a reference to a section of, or schedule, annex or exhibit to, this Agreement, and any reference to a sub-section, paragraph or sub-paragraph is a reference to the relevant provision within the Section, Schedule, Annex or Exhibit in which the reference appears;

(g)    the Recitals, Schedules, Annexes and Exhibits to this Agreement form an integral part of this Agreement and shall have the same force and effect as if set out in the body of this Agreement, and any reference to this Agreement shall include the Recitals, Schedules, Annexes and Exhibits;

(h)    the words "hereof", "herein", "hereto", "hereunder" and "hereby", and words of similar import, refer to this Agreement as a whole and not to any particular provision of this Agreement;

(i)    any reference to any statute, statutory provision, regulation, treaty, directive, code or other instrument having the force of law shall be construed as a reference to such statute, statutory provision, regulation, treaty, directive, code or instrument as in force from time to time and as amended, consolidated, modified, re-enacted or replaced (whether before or after the date of this Agreement), and shall include any subordinate legislation, rule or regulation made under it;

(j)    any reference to any document, agreement, deed or instrument (including this Agreement) is a reference to such document, agreement, deed or instrument as amended, novated, supplemented, varied, restated or replaced from time to time, save to the extent prohibited by such document, agreement, deed or instrument or by this Agreement;

(k)    any reference to a "day" means a calendar day, and any reference to a "month" or "year" means a calendar month or calendar year, in each case unless otherwise specified;

(l)    any reference to a "Business Day" has the meaning given in Section [Definitions], and where any payment or other act is required to be made or done on a day which is not a Business Day, such payment or act shall be made or done on the immediately succeeding Business Day;

(m)    in computing any period of time prescribed by, or allowed under, this Agreement, the day of the act or event from which the period of time begins to run shall be excluded and the last day of the period of time shall be included; and where a period of time is expressed in months or years, the period shall expire on the day in the relevant subsequent month or year corresponding numerically to the day from which the period began to run, or, if no such day exists in such month, on the last day of such month;

(n)    any reference to a time of day is a reference to the local time in [City, Country];

(o)    any reference to "writing" or "written" includes any method of reproducing words in a legible and non-transitory form, including email and other electronic communications, but excludes any other form of electronic display;

(p)    any reference to "[Currency]", "[Currency Symbol]" or any other currency shall be a reference to the lawful currency for the time being of the relevant jurisdiction of issue;

(q)    any reference to an "Affiliate" of any person means any other person that, directly or indirectly, through one or more intermediaries, Controls, is Controlled by, or is under common Control with, such first person, and "Control" (and its grammatical variants) means the possession, directly or indirectly, of the power to direct or cause the direction of the management or policies of a person, whether through the ownership of voting securities, by contract, by virtue of constitutional documents, or otherwise;

(r)    any reference to the "knowledge", "awareness", "belief" or "information" of a party (or any similar expression) shall mean the actual knowledge of the directors and senior officers of such party, after having made reasonable and due enquiry of those persons within such party's organisation who would, in the ordinary course, be expected to have knowledge of the relevant matter;

(s)    the words "material" and "materially" shall be construed by reference to the context in which they are used and, in the absence of any indication to the contrary, shall be construed by reference to this Agreement and the transactions contemplated by it as a whole;

(t)    any reference to a party's "consent", "approval", "agreement", "satisfaction" or "discretion" (or any similar expression) requires such consent, approval, agreement, satisfaction or discretion to be evidenced in writing and given in advance, unless expressly stated otherwise;

(u)    any reference to an English legal concept (including any reference to any action, remedy, method of judicial proceeding, document, legal status, court, official or any legal term of art) shall, in respect of any jurisdiction other than England, be deemed to include a reference to the concept which most nearly approximates in that jurisdiction to the English legal concept;

(v)    where any obligation is expressed to be performed or observed by two or more persons, the obligation of those persons shall (unless expressly stated to be several) be joint and several;

(w)    any phrase introduced by the words "other", "otherwise", "in particular", "for example", "such as" or "including" shall be construed as illustrative and shall not limit the generality of the related general words; and

(x)    general words shall not be given a restrictive meaning by reason of their being preceded or followed by words indicating a particular class of acts, matters or things.

[●].2    Each of the parties acknowledges that it has participated, with the benefit of independent legal advice, in the drafting and negotiation of this Agreement. Accordingly, the rule of construction known as the contra proferentem rule, and any other rule of construction to the effect that any ambiguity is to be resolved against the drafting party, shall not apply to the construction or interpretation of this Agreement or any provision hereof.

[●].3    The Parties intend this Agreement to take effect as a [deed/agreement under hand], and references to "execution", "executed" and similar expressions shall be construed accordingly.

[●].4    In the event of any conflict or inconsistency between the provisions of the main body of this Agreement and the provisions of any Schedule, Annex or Exhibit, the provisions of the main body of this Agreement shall prevail, save to the extent that the relevant Schedule, Annex or Exhibit expressly states that it is to prevail.

Rubric · gold standard

INTERPRETATION

This Agreement shall be interpreted in accordance with its terms, taking into account the principle of good faith, the economic purpose of its provisions, and the common intent of the Parties at the time of its execution.
The headings and titles used in this Agreement are included for convenience and reference only and shall not affect the interpretation of any provision of this Agreement.
Unless the context otherwise requires:
(a) words in the singular include the plural and vice versa;
(b) references to one gender include all genders;
(c) references to Sections, Clauses, Annexes or Schedules are references to sections, clauses, annexes or schedules of this Agreement;
(d) the words “including”, “includes” and “in particular” shall be construed as illustrative and shall not limit the generality of the preceding words;
(e) any reference to a law, regulation or legal provision shall include any amendment, replacement or re-enactment thereof; and
(f) capitalized terms used in this Agreement shall have the meanings assigned to them in this Agreement, whether defined in this Clause, in the preamble, in the recitals, in the body of this Agreement or in any Annex or Schedule.
This Agreement shall be deemed to have been jointly drafted and negotiated by the Parties, and no rule of construction or interpretation shall apply against any Party on the basis that such Party was responsible for drafting any provision of this Agreement.
In the event of any inconsistency between the body of this Agreement and any annexes or schedules attached hereto, the provisions of the main body of this Agreement shall prevail, unless expressly stated otherwise.

Obligation coverage

Obligation	Present	Evidence	Note
Consideration of good faith in interpretation	NO	—	The clause sets various interpretation rules but does not require interpretation to consider or be guided by good faith.
Consideration of the economic purpose of provisions in interpretation	YES	—	The candidate's interpretation rules are consistent with construing provisions in light of their economic and commercial purpose.
Consideration of the parties' common intent at execution in interpretation	NO	—	The clause sets general construction rules but does not state that interpretation should consider the parties' common intent at the time of execution.
Exclusion of headings and titles from interpretive effect	YES	“the headings and sub-headings to the Sections, Schedules, Annexes and Exhibits are inserted for convenience of reference only and shall not affect the construction or interpretation of this Agreement”	This expressly states that headings/sub-headings have no interpretive effect.
Reciprocal reading of singular and plural terms	YES	“words denoting the singular include the plural and vice versa”	This expressly provides the reciprocal singular/plural interpretation rule.
Gender-inclusive reading of references to one gender	YES	“words denoting any gender include all genders”	This expressly provides that references to one gender are to be read inclusively as covering all genders.
Reading of internal structural references as references within this agreement	YES	“any reference to a Section, Schedule, Annex or Exhibit is a reference to a section of, or schedule, annex or exhibit to, this Agreement”	This expressly provides that structural references are to provisions within this Agreement.
Illustrative non-limiting reading of expansive listing terms	YES	“the words "include", "includes" and "including" shall be construed as if followed by the words "without limitation"”	The clause expressly makes expansive listing terms illustrative and non-limiting.
Inclusion of amendments and replacements in legal references	YES	“any reference to any statute, statutory provision, regulation, treaty, directive, code or other instrument having the force of law shall be construed as a reference to such statute, statutory provision, regulation, treaty, directive, code or instrument as in force from time to time and as amended, consolidated, modified, re-enacted or replaced”	The clause expressly states that legal references include amended and replaced versions.
Apply assigned meanings to capitalized terms wherever defined in the agreement	NO	—	The clause defines a few specific terms and cross-references others, but it does not state a general rule that capitalized terms carry their assigned meanings wherever defined in the agreement.
Exclusion of drafter-based adverse construction against any party	YES	“the rule of construction known as the contra proferentem rule, and any other rule of construction to the effect that any ambiguity is to be resolved against the drafting party, shall not apply”	The clause expressly disapplies adverse construction against the drafting party.
Priority of the main body over annexes and schedules absent express override	YES	“In the event of any conflict or inconsistency between the provisions of the main body of this Agreement and the provisions of any Schedule, Annex or Exhibit, the provisions of the main body of this Agreement shall prevail, save to the extent that the relevant Schedule, Annex or Exhibit expressly states that it is to prevail.”	This expressly gives priority to the main body over schedules/annexes/exhibits unless an express override is stated.

Judge reasoning

The candidate is not fit for use because it omits three core interpretive directions expressly embodied in the reference clause: interpretation by reference to good faith, the economic purpose of the provisions, and the Parties' common intent at execution. In addition, it introduces numerous extra interpretive and substantive rules that materially affect rights and obligations beyond the reference, including deemed amendment of document references. Although much of the standard boilerplate is competently drafted, the omissions and additions mean it is not a reliable substitute for the reference in a live agreement.

Missing obligations

Consideration of good faith in interpretation
Consideration of the parties' common intent at execution in interpretation
Apply assigned meanings to capitalized terms wherever defined in the agreement

Hallucinations

Provision that recitals, schedules, annexes and exhibits form an integral part of the Agreement
Detailed time-computation rules for periods expressed in days, months or years
Definition of "Affiliate" and "Control"
Requirement that consent/approval/agreement/satisfaction/discretion be in writing and given in advance
Cross-jurisdiction substitution rule for English legal concepts
Joint-and-several liability rule for obligations of two or more persons
Statement that each party had the benefit of independent legal advice
Provision that the Agreement is intended to take effect as a deed/agreement under hand

Drafting issues

Clause [●].3 is an elective operative statement rather than a pure interpretation rule and is left uncompleted, creating uncertainty as to the Agreement's legal character.
The statement that each party participated with the benefit of independent legal advice may be factually inaccurate if not verified, creating avoidable risk.
Cross-reference to Section [Definitions] is not usable as drafted unless that section exists and is correctly numbered.