跳转至

Week7-Lecture16-18

Lecture 16 Iterators

Reading 4.2 Implicit Sequences 隐式序列

核心思想:序列不一定要把每个元素都提前算好、存在内存里,可以"按需计算"(惰性计算,lazy computation)

4.2.1 Iterators

迭代器是一种"只能往前走一步一步取值"的对象,核心是两个内置函数:

  • iter(容器) → 从一个容器(如列表)得到一个迭代器
  • next(迭代器) → 取下一个值,取完了就抛出 StopIteration 异常
Python
1
2
3
4
nums = [2, 3, 5, 7]
it = iter(nums)
next(it)  # 2
next(it)  # 3
  • 迭代器有状态,记着自己走到哪了。对同一个容器调用两次 iter(),会得到两个互不影响、各自独立计数的迭代器。
  • 但如果你把同一个迭代器赋给两个变量名(比如 u = t),这两个名字指向的是同一个迭代器,一个走了另一个也跟着走。
  • 对一个迭代器再调用 iter(),会返回它自己(不是复制品)。这样设计是为了让你不用管手上的东西是"容器"还是"迭代器",统一用 iter() 包一层就行。

4.2.2 Iterable

能被iter()接受的东西就叫可迭代对象。即使是字典这种无序的集合,也可以被迭代。

Python
>>> d = {'one': 1, 'two': 2, 'three': 3}
>>> d
{'one': 1, 'three': 3, 'two': 2}
>>> k = iter(d)
>>> next(k)
'one'
>>> next(k)
'three'
>>> v = iter(d.values())
>>> next(v)
1
>>> next(v)
3

>>> d.pop('two')
2
>>> next(k) #字典结构变化了
       
RuntimeError: dictionary changed size during iteration
Traceback (most recent call last):

4.2.3 Built-in Iterators

map、filter、zip、reversed 这些内置函数,在 Python 3 里返回的都是迭代器,而不是直接算出结果的列表。也就是说 map(f, s) 这一行代码本身不会去调用 f,只有你 next() 它或者用 list() 强制展开它的时候,f 才真正被逐个调用。这就是"惰性"在内置函数上的体现——调用和真正的计算是分离的。

Python
>>> def double_and_print(x):
        print('***', x, '=>', 2*x, '***')
        return 2*x
>>> s = range(3, 7)
>>> doubled = map(double_and_print, s)  # double_and_print not yet called
>>> next(doubled)                       # double_and_print called once
*** 3 => 6 ***
6
>>> next(doubled)                       # double_and_print called again
*** 4 => 8 ***
8
>>> list(doubled)                       # double_and_print called twice more
*** 5 => 10 ***
*** 6 => 12 ***
[10, 12]

4.2.4 For Statements

for x in 容器: 这个语法,本质上是 Python 帮你自动做了这几步:

  1. 对容器调用 __iter__() 得到一个迭代器
  2. 反复调用这个迭代器的 __next__(),把结果赋给 x
  3. 直到捕获到 StopIteration 就自动结束循环(这个异常被悄悄处理掉,不会报错)
Python
1
2
3
4
5
6
7
it = counts.__iter__()
try:
    while True:
        item = it.__next__()
        print(item)
except StopIteration:
    pass

4.2.5 Yield

自己动手写迭代器很麻烦,生成器用于用普通函数的写法定义一个迭代器,不用自己管理状态。
区别在于:普通函数用 return 交出结果并结束;生成器函数用 yield 交出一个值,但函数会"暂停"在这里,状态全部保留,等下次 __next__() 被调用时,从暂停的地方接着往下执行。

Python
>>> def letters_generator():
        current = 'a'
        while current <= 'd':
            yield current
            current = chr(ord(current)+1)

>>> for letter in letters_generator():
        print(letter)
a
b
c
d

>>> letters = letters_generator()
>>> type(letters)
<class 'generator'>
>>> letters.__next__()
'a'
>>> letters.__next__()
'b'
>>> letters.__next__()
'c'
>>> letters.__next__()
'd'
>>> letters.__next__()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

4.2.6 Iterable Interface

  • 可迭代对象 = 数据本身的"容器",不会变。每次调用它的 __iter__() 都能产出一个全新的迭代器。
  • 迭代器 = 遍历过程中的"进度指针",会随着 next() 不断变化。

4.2.7 Creating Iterables with Yield

Python
# 需要多次遍历如何解决
>>> def all_pairs(s):
        for item1 in s:
            for item2 in s:
                yield (item1, item2)

>>> list(all_pairs([1, 2, 3])) #[1, 2, 3]是可迭代对象,不是迭代器,每次__iter__()都会生成新的迭代器
[(1, 1), (1, 2), (1, 3), (2, 1), (2, 2), (2, 3), (3, 1), (3, 2), (3, 3)]

# 另一种方法就是在定义对象的时候就加上去
>>> class LettersWithYield:
        def __init__(self, start='a', end='e'):
            self.start = start
            self.end = end
        def __iter__(self):
            next_letter = self.start
            while next_letter < self.end:
                yield next_letter
                next_letter = chr(ord(next_letter)+1)

4.2.8 Iterable Interface

反过来,如果你想手写一个不借助 yield 的迭代器类,核心只需要实现 __next__:每次调用返回下一个元素,并且在对象内部更新自身状态,状态耗尽时 raise StopIteration。

因为你可以完全自己控制什么时候算"结束",迭代器天然能表示无穷序列——只要 __next__ 永远不抛异常就行,比如一个不断 +1 吐出正整数的 Positives 类。这是隐式序列相比"提前存好的列表"最大的优势之一。

4.2.9 / 4.2.10 Stream

Stream(流)是另一种表示隐式序列的方式,可以理解成惰性版的链表:普通链表 Link(first, rest) 里 rest 是提前算好、直接存着的另一个链表;而 Stream 里的 rest 是一个还没算的函数,只有你真的去访问 .rest 属性时,才会调用这个函数把结果算出来,而且算出来后会缓存下来,不会重复计算第二次。

这带来一个和迭代器本质不同的特性:流可以被多次、反复地传给不同的纯函数处理,每次结果都一样,不会像迭代器那样"用一次就耗尽"。

Iterators, generators, streams共同解决一个无穷大的序列如何表示。

Lab 05 Iterators, Mutability

Python
SOURCE_FILE = __file__


def insert_items(s: list[int], before: int, after: int) -> list[int]:
    """Insert after into s following each occurrence of before and then return s.

    >>> test_s = [1, 5, 8, 5, 2, 3]
    >>> new_s = insert_items(test_s, 5, 7)
    >>> new_s
    [1, 5, 7, 8, 5, 7, 2, 3]
    >>> test_s
    [1, 5, 7, 8, 5, 7, 2, 3]
    >>> new_s is test_s
    True
    >>> double_s = [1, 2, 1, 2, 3, 3]
    >>> double_s = insert_items(double_s, 3, 4)
    >>> double_s
    [1, 2, 1, 2, 3, 4, 3, 4]
    >>> large_s = [1, 4, 8]
    >>> large_s2 = insert_items(large_s, 4, 4)
    >>> large_s2
    [1, 4, 4, 8]
    >>> large_s3 = insert_items(large_s2, 4, 6)
    >>> large_s3
    [1, 4, 6, 4, 6, 8]
    >>> large_s3 is large_s
    True
    """
    index_list = []
    for i in range(len(s)):
        if s[i] == before:
            index_list.append(i)
    for j in range(len(index_list)):
        index_list[j] += j
    for k in index_list:
        s.insert(k + 1, after)
    return s


def group_by(s: list[int], fn) -> dict[int, list[int]]:
    """Return a dictionary of lists that together contain the elements of s.
    The key for each list is the value that fn returns when called on any of the
    values of that list.

    >>> group_by([12, 23, 14, 45], lambda p: p // 10)
    {1: [12, 14], 2: [23], 4: [45]}
    >>> group_by(range(-3, 4), lambda x: x * x)
    {9: [-3, 3], 4: [-2, 2], 1: [-1, 1], 0: [0]}
    """
    grouped = {}
    for i in s:
        key = fn(i)
        if key in grouped:
            grouped[key].append(i)
        else:
            grouped[key] = [i]
    return grouped


def sprout_leaves(t, leaves):
    """Sprout new leaves containing the labels in leaves at each leaf of
    the original tree t and return the resulting tree.

    >>> t1 = tree(1, [tree(2), tree(3)])
    >>> print_tree(t1)
    1
      2
      3
    >>> new1 = sprout_leaves(t1, [4, 5])
    >>> print_tree(new1)
    1
      2
        4
        5
      3
        4
        5

    >>> t2 = tree(1, [tree(2, [tree(3)])])
    >>> print_tree(t2)
    1
      2
        3
    >>> new2 = sprout_leaves(t2, [6, 1, 2])
    >>> print_tree(new2)
    1
      2
        3
          6
          1
          2
    """
    if is_leaf(t):
        return tree(label(t), [tree(leave) for leave in leaves])
    new_branches = [sprout_leaves(b, leaves) for b in branches(t)]
    return tree(label(t), new_branches)


from typing import Iterator  # "t: Iterator[int]" means t is an iterator that yields integers

def count_occurrences(t: Iterator[int], n: int, x: int) -> int:
    """Return the number of times that x is equal to one of the
    first n elements of iterator t.

    >>> s = iter([10, 9, 10, 9, 9, 10, 8, 8, 8, 7])
    >>> count_occurrences(s, 10, 9)
    3
    >>> t = iter([10, 9, 10, 9, 9, 10, 8, 8, 8, 7])
    >>> count_occurrences(t, 3, 10)
    2
    >>> u = iter([3, 2, 2, 2, 1, 2, 1, 4, 4, 5, 5, 5])
    >>> count_occurrences(u, 1, 3)  # Only iterate over 3
    1
    >>> count_occurrences(u, 3, 2)  # Only iterate over 2, 2, 2
    3
    >>> list(u)                     # Ensure that the iterator has advanced the right amount
    [1, 2, 1, 4, 4, 5, 5, 5]
    >>> v = iter([4, 1, 6, 6, 7, 7, 6, 6, 2, 2, 2, 5])
    >>> count_occurrences(v, 6, 6)
    2
    """
    count = 0
    while n > 0:
        if next(t) == x:
            count += 1
        n -= 1
    return count


def pathsum(t, n):
    """
    >>> my_tree = tree(2, [tree(3, [tree(5), tree(7)]), tree(4)])
    >>> pathsum(my_tree, 12) # 2 -> 3 -> 7
    True
    >>> pathsum(my_tree, 5)  # A path that doesn't reach a leaf such as 2 -> 3 doesn't count
    False
    """
    if n == label(t) and is_leaf(t):
        return True
    return any([pathsum(b, n-label(t)) for b in branches(t)])


def sum_tree(t):
    """Add all elements in a tree.

    >>> t = tree(4, [tree(2, [tree(3)]), tree(6)])
    >>> sum_tree(t)
    15
    """
    # total = 0
    # for b in branches(t):
    #     total += sum_tree(b)
    # return total + label(t)
    if is_leaf(t):
        return label(t)
    return label(t) + sum([sum_tree(b) for b in branches(t)])

def balanced(t):
    """Checks if each branch has same sum of all elements and
    if each branch is balanced.

    >>> t = tree(1, [tree(3), tree(1, [tree(2)]), tree(1, [tree(1), tree(1)])])
    >>> balanced(t)
    True
    >>> t = tree(1, [t, tree(1)])
    >>> balanced(t)
    False
    >>> t = tree(1, [tree(4), tree(1, [tree(2), tree(1)]), tree(1, [tree(3)])])
    >>> balanced(t)
    False
    """
    for b in branches(t):
        if sum_tree(branches(t)[0]) != sum_tree(b) or not balanced(b):
            return False
    return True




# Tree Data Abstraction

def tree(label, branches=[]):
    """Construct a tree with the given label value and a list of branches."""
    for branch in branches:
        assert is_tree(branch), 'branches must be trees'
    return [label] + list(branches)

def label(tree):
    """Return the label value of a tree."""
    return tree[0]

def branches(tree):
    """Return the list of branches of the given tree."""
    return tree[1:]

def is_tree(tree):
    """Returns True if the given tree is a tree, and False otherwise."""
    if type(tree) != list or len(tree) < 1:
        return False
    for branch in branches(tree):
        if not is_tree(branch):
            return False
    return True

def is_leaf(tree):
    """Returns True if the given tree's list of branches is empty, and False
    otherwise.
    """
    return not branches(tree)

def print_tree(t, indent=0):
    """Print a representation of this tree in which each node is
    indented by two spaces times its depth from the root.

    >>> print_tree(tree(1))
    1
    >>> print_tree(tree(1, [tree(2)]))
    1
      2
    >>> numbers = tree(1, [tree(2), tree(3, [tree(4), tree(5)]), tree(6, [tree(7)])])
    >>> print_tree(numbers)
    1
      2
      3
        4
        5
      6
        7
    """
    print('  ' * indent + str(label(t)))
    for b in branches(t):
        print_tree(b, indent + 1)

def copy_tree(t):
    """Returns a copy of t. Only for testing purposes.

    >>> t = tree(5)
    >>> copy = copy_tree(t)
    >>> t = tree(6)
    >>> print_tree(copy)
    5
    """
    return tree(label(t), [copy_tree(b) for b in branches(t)])

Lecture 17 Generators

Disc 06 Iterators, Generators

Python
def gen_fib():
    n, add = 0, 1
    while True:
        yield n
        n, add = n + add, n

next(filter(lambda n: n > 2026, gen_fib())) 

def repeated(t, k):
    """Return the first value in iterator t that appears k times in a row,
    calling next on t as few times as possible.

    >>> s = iter([10, 9, 10, 9, 9, 10, 8, 8, 8, 7])
    >>> repeated(s, 2)
    9
    >>> t = iter([10, 9, 10, 9, 9, 10, 8, 8, 8, 7])
    >>> repeated(t, 3)
    8
    >>> u = iter([3, 2, 2, 2, 1, 2, 1, 4, 4, 5, 5, 5])
    >>> repeated(u, 3)
    2
    >>> repeated(u, 3)
    5
    >>> v = iter([4, 1, 6, 6, 7, 7, 8, 8, 2, 2, 2, 5])
    >>> repeated(v, 3)
    2
    """
    assert k > 1
    count = 0
    last_item = None
    while True:
        item = next(t)
        if item == last_item:
            count += 1
        else:
            last_item = item
            count = 1
        if count == k:
            return item

def differences(t):
    """Yield the differences between adjacent values from iterator t.

    >>> list(differences(iter([5, 2, -100, 103])))
    [-3, -102, 203]
    >>> next(differences(iter([39, 100])))
    61
    """
    last_x = next(t)
    for x in t:
        yield x - last_x
        last_x = x 


def partition_gen(n, m):
    """Yield the partitions of n using parts up to size m.
    >>> for partition in sorted(partition_gen(6, 4)):
    ...     print(partition)
    1 + 1 + 1 + 1 + 1 + 1
    1 + 1 + 1 + 1 + 2
    1 + 1 + 1 + 3
    1 + 1 + 2 + 2
    1 + 1 + 4
    1 + 2 + 3
    2 + 2 + 2
    2 + 4
    3 + 3
    """
    assert n > 0 and m > 0
    if n == m:
        yield str(n)
    if n - m > 0:
        for p in partition_gen(n-m, m):
            yield p + ' + ' + str(m)
    if m > 1:
        yield from partition_gen(n, m-1)

def squares(total, k):
    """Yield the ways in which perfect squares greater or equal to k*k sum to total.

    >>> list(squares(10, 1))  # All lists of perfect squares that sum to 10
    [[1, 1, 1, 1, 1, 1, 1, 1, 1, 1], [4, 1, 1, 1, 1, 1, 1], [4, 4, 1, 1], [9, 1]]
    >>> list(squares(20, 2))  # Only use perfect squares greater or equal to 4 (2*2).
    [[4, 4, 4, 4, 4], [16, 4]]
    """
    assert total > 0 and k > 0
    if total == k * k:
        yield [k * k]
    elif total > k * k:
        for s in squares(total - k * k, k):
            yield s + [k * k]
        yield from squares(total, k + 1)


def church_generator(f):
    """Takes in a function f and yields functions which apply f
    to their argument one more time than the previously generated
    function.

    >>> increment = lambda x: x + 1
    >>> church = church_generator(increment)
    >>> for _ in range(5):
    ...     fn = next(church)
    ...     print(fn(0))
    0
    1
    2
    3
    4
    """

    g = lambda x: x
    while True:
        yield g 
        g = (lambda g: lambda x: f(g(x)))(g)

Lecture 18 Object

Reading 2.5 Object-Oriented Programming

OOP是一种组织程序的技术。汇聚了第二章中许多思想(抽象屏障、行为请求、局部状态)。
  It enables a new metaphor for designing programs in which several independent agents interact within the computer.
  对象是一种数据值,具有可通过点表示法访问的方法和属性。每个对象还都有一个类型,称为它的类。要创建新的数据类型,我们就实现新的类。

2.5.1 Objects and Classes

类是对象的模版,对象是类的实例。类定义了属性和方法。

Python
#一个对象应该实现
>>> a = Account('Kirk')
>>> a.holder
'Kirk'
>>> a.deposit(15)
15
>>> a.withdraw(10)
5
>>> a.withdraw(10)
'Insufficient funds' # 对象内部是有状态的

2.5.2 Defining Classes

Python
class <name>:
    <suite>
Python
class Account:
    def __init__(self, account_holder): 
    #__init__构造函数,创建实例时自动调用,用来初始化实例属性。
    #第一个参数总是self,代表"正在被创建/操作的那个对象",这是约定俗成的命名(不是语法强制,但几乎所有 Python 代码都遵守)。
        self.balance = 0
        self.holder = account_holder
    def deposit(self, amount):
        self.balance = self.balance + amount
        return self.balance
    def withdraw(self, amount):
        if amount > self.balance:
            return 'Insufficient funds'
        self.balance = self.balance - amount
        return self.balance

>>> a = Account('Kirk')
>>> b = Account('Spock')
>>> b.balance = 200
>>> a is a        # True
>>> a is not b     # True(虽然都是 Account,但是不同对象)
>>> c = a
>>> c is a         # True(赋值只是绑定同一个对象,没有复制)

2.5.3 Message Passing and Dot Expressions

Python
1
2
3
4
# 函数与方法的区别
>>> type(Account.deposit)        # <class 'function'>——从类上取,是普通函数
>>> type(spock_account.deposit)  # <class 'method'>——从实例上取,是"绑定方法"
# Account.deposit 需要手动传入 self;而 spock_account.deposit 已经把 spock_account 自动"绑定"为第一个参数了

命名约定:类名用 CapWords(如 CheckingAccount),方法名用小写+下划线;以下划线开头的属性(如 _balance)表示"这是实现细节,外部不应直接访问"。

2.5.4 Class Attributes

  • 直接改类属性(Account.interest = 0.04)会影响所有没有同名实例属性的实例;
  • 给某个实例单独赋值(kirk_account.interest = 0.08)只会创建一个新的实例属性,遮盖同名类属性,不会影响类或其他实例。
Python
>>> kirk_account.interest = 0.08   # 创建实例属性,只影响 kirk_account
>>> Account.interest = 0.05        # 改类属性,影响 spock_account,但不影响 kirk_account

2.5.5 Inheritance

当两个类"很像,但一个是另一个的特殊情形"时,用继承避免重复代码。

  • 术语:基类/父类(base/super class) vs 子类(subclass)
  • 子类继承基类的全部属性,也可以**覆盖(override)**部分属性/方法
  • 继承表达的是 is-a 关系("支票账户是一种账户"),而不是 has-a 关系

2.5.6 Using Inheritance

Python
1
2
3
4
5
6
class CheckingAccount(Account):     # 括号里写基类
    """A bank account that charges for withdrawals."""
    withdraw_charge = 1
    interest = 0.01
    def withdraw(self, amount):
        return Account.withdraw(self, amount + self.withdraw_charge)

名字查找的递归规则(重要):在实例上找不到某名字时,Python 会依次到"实例所属的类 → 基类 → 基类的基类…"中查找。

接口(Interface):好的代码应该只依赖"对象有没有某个方法/属性",而不假设对象的具体类型。

2.5.7 Multiple Inheritance

Python
1
2
3
4
class AsSeenOnTVAccount(CheckingAccount, SavingsAccount):
    def __init__(self, account_holder):
        self.holder = account_holder
        self.balance = 1

多个基类都定义了同名属性/方法:从左到右、再往上的查找顺序

2.5.8 The role of object

  • 对象系统的目的是让**关注点分离(separation of concerns)**更容易:每个对象封装自己的一部分状态,每个类实现程序逻辑的一部分。
  • OOP 特别适合"多个独立个体相互交互"的系统建模(社交网络里的用户、游戏里的角色、物理模拟里的形状等)。
  • 但不是所有逻辑都该塞进类里——如果用函数表达"输入到输出的映射关系"更自然,就该用函数。函数同样可以做到关注点分离。
  • Python 是多范式语言,该用类还是用函数,是需要培养的设计判断力。

Homework 05

Python
def hailstone(n):
    """
    Yields the elements of the hailstone sequence starting at n.
    At the end of the sequence, yield 1 infinitely.

    >>> hail_gen = hailstone(10)
    >>> [next(hail_gen) for _ in range(10)]
    [10, 5, 16, 8, 4, 2, 1, 1, 1, 1]
    >>> next(hail_gen)
    1
    """
    yield n
    if n == 1:
        yield from hailstone(n)
    elif n % 2 == 0:
        yield from hailstone(n//2)
    else:
        yield from hailstone(3 * n + 1)


def merge(a, b):
    """
    Return a generator that has all of the elements of infinite iterators a and b,
    in increasing order, without duplicates.

    >>> def sequence(start, step):
    ...     while True:
    ...         yield start
    ...         start += step
    >>> a = sequence(2, 3) # 2, 5, 8, 11, 14, ...
    >>> b = sequence(3, 2) # 3, 5, 7, 9, 11, 13, 15, ...
    >>> result = merge(a, b) # 2, 3, 5, 7, 8, 9, 11, 13, 14, 15
    >>> [next(result) for _ in range(10)]
    [2, 3, 5, 7, 8, 9, 11, 13, 14, 15]
    """
    a_val, b_val = next(a), next(b)
    while True:
        if a_val == b_val:
            yield a_val
            a_val, b_val = next(a), next(b)

        elif a_val < b_val:
            yield a_val
            a_val = next(a)
        else:
            yield b_val
            b_val = next(b)


def stair_ways(n):
    """
    Yield all the ways to climb a set of n stairs taking
    1 or 2 steps at a time.

    >>> list(stair_ways(0))
    [[]]
    >>> s_w = stair_ways(4)
    >>> sorted([next(s_w) for _ in range(5)])
    [[1, 1, 1, 1], [1, 1, 2], [1, 2, 1], [2, 1, 1], [2, 2]]
    >>> list(s_w) # Ensure you're not yielding extra
    []
    """
    if n == 0:
        yield []
    elif n == 1:
        yield [1]
    else:
        for way in stair_ways(n-1):
            yield way + [1]
        for way in stair_ways(n-2):
            yield way + [2]


def yield_paths(t, target):
    """
    Yields all possible paths from the root of t to a node with the label
    target as a list.

    >>> t1 = tree(1, [tree(2, [tree(3), tree(4, [tree(6)]), tree(5)]), tree(5)])
    >>> print_tree(t1)
    1
      2
        3
        4
          6
        5
      5
    >>> next(yield_paths(t1, 6))
    [1, 2, 4, 6]
    >>> path_to_5 = yield_paths(t1, 5)
    >>> sorted(list(path_to_5))
    [[1, 2, 5], [1, 5]]

    >>> t2 = tree(0, [tree(2, [t1])])
    >>> print_tree(t2)
    0
      2
        1
          2
            3
            4
              6
            5
          5
    >>> path_to_2 = yield_paths(t2, 2)
    >>> sorted(list(path_to_2))
    [[0, 2], [0, 2, 1, 2]]
    """
    if label(t) == target:
        yield [label(t)]
    for b in branches(t):
        for k in yield_paths(b, target):
            yield [label(t)] + k 

Project Ants

Project3-Ants