0

I'm learning Rust and am trying to write a simple lexer. I have a function next_token in an impl block that reads the next token from an input character iterator. However, the code will not build because of multiple borrows of self. How do I get around this?

struct Lexer<'a> {
    input: Peekable<Chars<'a>>
}

...

impl<'a> Lexer<'a> {
    pub fn create(input: &str) -> Lexer {
        Lexer { input: input.chars().peekable() }
    }

    fn next_token(&mut self) -> Option<Token> {
        match self.input.by_ref().peek() {
            Some(c) if c.is_alphabetic() || *c == '_' => { // Identifier or keyword
                let literal = self.input.take_while(|&ch| {
                    match self.input.peek() {
                        Some(inner_c) => inner_c.is_alphanumeric() || *inner_c == '_',
                        None => false
                    }
                }).collect::<String>();
                if let Some(keyword) = check_keyword(&literal) { // Literal is a keyword
                    return Some(Token { literal: literal, token_type: keyword });
                } else { // Literal is identifier
                    return Some(Token { literal: literal, token_type: TokenType::Identifier });
                };
            },
            ... // Other cases
        }
    }
}

The error I am getting:

error[E0507]: cannot move out of borrowed content --> lexer.rs:110:20 | 110 | let literal = self.input.take_while(|&ch| { | ^^^^ cannot move out of borrowed content

error[E0500]: closure requires unique access to self but self.input is already borrowed --> lexer.rs:110:42 | 108 | match self.input.by_ref().peek() { | ---------- borrow occurs here 109 | Some(c) if c.is_alphabetic() || *c == '_' => { // Identifier or keyword 110 | let literal = self.input.take_while(|&ch| { | ^^^^^ closure construction occurs here 111 |
match self.input.peek() { | ---- borrow occurs due to use of self in closure ... 139 | } | - borrow ends here

Shepmaster
  • 388,571
  • 95
  • 1,107
  • 1,366
GregoryComer
  • 740
  • 8
  • 18
  • Can't find appropriate duplicate at the moment, but it exists. Split up your type into more fine-grained types and move the methods to those. Then you can call `self.thing1.fn1(...)` `self.thing2.fn2(...)`. – Shepmaster Oct 31 '16 at 23:00
  • Can I do that if I'm referencing self.input in both locations? – GregoryComer Oct 31 '16 at 23:06
  • 1
    Have to see code to be sure, but I believe so. If you do `self.thing1.fn(self.input)` and `self.thing2.fn(self.input)`, for example. Short version is that borrow checking stops at the function boundary. Inside one function, it can see that `self.thing1` and `self.input` are disjoint, but if you borrow `self.input` in one function and call another method on `self, then it assumes that method could do anything, including invalidate your mutable reference. – Shepmaster Oct 31 '16 at 23:54
  • 1
    You can circumvent your specific problem by using a [more cautious take_while](http://stackoverflow.com/questions/28776630/implementing-a-cautious-take-while-using-peekable) – Sebastian Ullrich Nov 01 '16 at 02:52
  • Thanks. It looks like that's what I need. – GregoryComer Nov 01 '16 at 03:55

0 Answers0