2

I have a simple network:

input_layer = Input(1)
inner_layer = Dense(4, activation='relu')(input_layer)
output_layer = Dense(1, activation='linear')(inner_layer)
model = Model(input_layer, output_layer)
optimizer = Adam(learning_rate=0.01)
model.compile(optimizer=optimizer, loss='mse')

Intuitively, inference for input 0 would simply be model.predict(0). However this generates this error: expected input_2 to have 2 dimensions, but got array with shape ()

I understand it expects the input (which is a single number) to be two dimensional, but I don't understand what Tensorflow accepts as valid input. I tried many different combinations of inputs, some work and some don't, it seems quite inconsistent and the warnings/error are usually not useful:

When calling model.predict():

  • model.predict(0) - Throws
  • model.predict([0]) - Works
  • model.predict([[0]]) - Works

When calling model() (I saw here that's needed to get the gradients):

  • model(0) - Throws
  • model([0]) - Throws
  • model([[0]]) - Throws

When using np.reshape:

  • model(np.reshape(0,[1,1])) - Works
  • model(np.reshape([0],[1,1])) - Works
  • model(np.reshape([[0]],[1,1])) - Works

What seems to be working consistently is using numpy's reshape function. it always works both for model.predict() and for model() on all the inputs as long as they're reshaped to a [1,1] shape.

My questions:

  1. What are the guidelines to feeding inputs into tensorflow models in regards to inputs shapes/types?
  2. What does "shape ()" mean?
  3. What does "(None, 1)" mean?
  4. Why does reshape work but [[0]] does not? Both create a 2-dimensional collection.
  5. Why when calling model(0)/model([0])/model([[0]]) does this warning show: WARNING:tensorflow:Model was constructed with shape Tensor("input_1:0", shape=(None, 1), dtype=float32) for input (None, 1), but it was re-called on a Tensor with incompatible shape ()?
asaf92
  • 1,557
  • 1
  • 19
  • 30

1 Answers1

4

The shape of the tensor inputs = tf.keras.layers.Input(1) is (None, 1) (run inputs.get_shape().as_list()). The None means any size that is determined dynamically (batch size). The 1 is the shape of your data point. For example, this is a tensor of shape (3, 1):

[[1], [2], [1]]

This is a tensor of shape (3,)

[1, 2, 1]

If you define a tensor of shape (None, 1) you must feed the data of same shape.

The [[0]] has correct shape (1, 1) and won't throw any error or warning if you pass it as numpy array of expected data type:

import tensorflow as tf
import numpy as np

input_layer = tf.keras.layers.Input(1)
inner_layer = tf.keras.layers.Dense(4, activation='relu')(input_layer)
output_layer = tf.keras.layers.Dense(1, activation='linear')(inner_layer)
model = tf.keras.models.Model(input_layer, output_layer)
optimizer = tf.keras.optimizers.Adam(learning_rate=0.01)
model.compile(optimizer=optimizer, loss='mse')
print(model(np.array([[0.]], dtype=np.float32)).numpy()) # [[0.]]
print(model.predict(np.array([[0.], [1]], dtype=np.float32))) # [[0.        ]
                                                              # [0.08964952]]

np.reshape() works because it automatically converts your list to numpy array. For more about np.reshape refer to the official documentation.

The model.predict() also expects the same shape as model.__call__(), but can perform automatic reshaping (expands dimension on the left, i.e [1] -- > [[1]]).

Vlad
  • 8,225
  • 5
  • 33
  • 45